ó î&]\c @`s›dZddlmZmZmZddlmZddlmZddl Z ddl m Z ddd d d d gZ defd „ƒYZdddd$dddddd„ Zddddddddd„ZeZdddd%dddddd„ Zdddddddddd„ Zd„Zd„Zd„Zd&ddd„Zd'ddd„Zdddd„Zddd d!„Zddd d"d#„ZdS((s· Functions --------- .. autosummary:: :toctree: generated/ line_search_armijo line_search_wolfe1 line_search_wolfe2 scalar_search_wolfe1 scalar_search_wolfe2 i(tdivisiontprint_functiontabsolute_import(twarn(tminpack2N(txrangetLineSearchWarningtline_search_wolfe1tline_search_wolfe2tscalar_search_wolfe1tscalar_search_wolfe2tline_search_armijocB`seZRS((t__name__t __module__(((s8lib/python2.7/site-packages/scipy/optimize/linesearch.pyRsg-Cëâ6?gÍÌÌÌÌÌì?i2g:Œ0âŽyE>g›+¡†›„=c `s0|d krˆˆ ƒ}ntˆtƒrWˆd} ˆd‰ˆ| fˆ‰t‰n ˆ‰t‰|g‰dg‰dg‰‡‡‡‡‡ fd†}‡‡‡‡‡‡‡‡ fd†}tj|ˆƒ}t|||||d|d| d| d| d | ƒ\}}}|ˆdˆd||ˆdfS( sÞ As `scalar_search_wolfe1` but do a line search to direction `pk` Parameters ---------- f : callable Function `f(x)` fprime : callable Gradient of `f` xk : array_like Current point pk : array_like Search direction gfk : array_like, optional Gradient of `f` at point `xk` old_fval : float, optional Value of `f` at point `xk` old_old_fval : float, optional Value of `f` at point preceding `xk` The rest of the parameters are the same as for `scalar_search_wolfe1`. Returns ------- stp, f_count, g_count, fval, old_fval As in `line_search_wolfe1` gval : array Gradient of `f` at the final point iic`s%ˆdcd7<ˆˆ|ˆˆŒS(Nii((ts(targstftfctpktxk(s8lib/python2.7/site-packages/scipy/optimize/linesearch.pytphiUsc`sbˆˆ|ˆˆŒˆd<ˆr4ˆdcd7 0 is assumed to be a descent direction. Parameters ---------- phi : callable phi(alpha) Function at point `alpha` derphi : callable dphi(alpha) Derivative `d phi(alpha)/ds`. Returns a scalar. phi0 : float, optional Value of `f` at 0 old_phi0 : float, optional Value of `f` at the previous point derphi0 : float, optional Value `derphi` at 0 c1, c2 : float, optional Wolfe parameters amax, amin : float, optional Maximum and minimum step size xtol : float, optional Relative tolerance for an acceptable step. Returns ------- alpha : float Step size, or None if no suitable step was found phi : float Value of `phi` at the new point `alpha` phi0 : float Value of `phi` at `alpha=0` Notes ----- Uses routine DCSRCH from MINPACK. gigð?g)\Âõ(ð?ii tSTARTidtFGitERRORitWARNNg)\Âõ(@(i(i ( R#tminRtzerostintctfloatRRtdcsrch(RRtphi0told_phi0R,RRR R!R"talpha1tphi1tderphi1tisavetdsavettasktmaxitertiR-((s8lib/python2.7/site-packages/scipy/optimize/linesearch.pyR js8+        i c  `sydg‰dg‰dg‰dg‰‡‡‡‡ ‡ fd†} tˆ tƒru‡‡‡‡‡‡ ‡ ‡ fd†‰n'ˆ ‰‡‡‡‡‡‡ ‡ fd†‰|dkrºˆˆ ˆŒ}ntj|ˆ ƒ}ˆdk rù‡‡‡‡‡ ‡ fd†}nd}t| ˆ||||| | |d| ƒ \}}}}|dkrQtdtƒn ˆd}|ˆdˆd|||fS( s Find alpha that satisfies strong Wolfe conditions. Parameters ---------- f : callable f(x,*args) Objective function. myfprime : callable f'(x,*args) Objective function gradient. xk : ndarray Starting point. pk : ndarray Search direction. gfk : ndarray, optional Gradient value for x=xk (xk being the current parameter estimate). Will be recomputed if omitted. old_fval : float, optional Function value for x=xk. Will be recomputed if omitted. old_old_fval : float, optional Function value for the point preceding x=xk args : tuple, optional Additional arguments passed to objective function. c1 : float, optional Parameter for Armijo condition rule. c2 : float, optional Parameter for curvature condition rule. amax : float, optional Maximum step size extra_condition : callable, optional A callable of the form ``extra_condition(alpha, x, f, g)`` returning a boolean. Arguments are the proposed step ``alpha`` and the corresponding ``x``, ``f`` and ``g`` values. The line search accepts the value of ``alpha`` only if this callable returns ``True``. If the callable returns ``False`` for the step length, the algorithm will continue with new iterates. The callable is only called for iterates satisfying the strong Wolfe conditions. maxiter : int, optional Maximum number of iterations to perform Returns ------- alpha : float or None Alpha for which ``x_new = x0 + alpha * pk``, or None if the line search algorithm did not converge. fc : int Number of function evaluations made. gc : int Number of gradient evaluations made. new_fval : float or None New function value ``f(x_new)=f(x0+alpha*pk)``, or None if the line search algorithm did not converge. old_fval : float Old function value ``f(x0)``. new_slope : float or None The local slope along the search direction at the new value ````, or None if the line search algorithm did not converge. Notes ----- Uses the line search algorithm to enforce strong Wolfe conditions. See Wright and Nocedal, 'Numerical Optimization', 1999, pg. 59-60. For the zoom phase it uses an algorithm by [...]. ic`s%ˆdcd7<ˆˆ|ˆˆŒS(Nii((talpha(RRRRR(s8lib/python2.7/site-packages/scipy/optimize/linesearch.pyRsc`swˆdctˆƒd7<ˆd}ˆd}ˆ|fˆ}|ˆ|ˆ|Œˆd<|ˆd 0 is assumed to be a descent direction. Parameters ---------- phi : callable f(x) Objective scalar function. derphi : callable f'(x), optional Objective function derivative (can be None) phi0 : float, optional Value of phi at s=0 old_phi0 : float, optional Value of phi at previous point derphi0 : float, optional Value of derphi at s=0 c1 : float, optional Parameter for Armijo condition rule. c2 : float, optional Parameter for curvature condition rule. amax : float, optional Maximum step size extra_condition : callable, optional A callable of the form ``extra_condition(alpha, phi_value)`` returning a boolean. The line search accepts the value of ``alpha`` only if this callable returns ``True``. If the callable returns ``False`` for the step length, the algorithm will continue with new iterates. The callable is only called for iterates satisfying the strong Wolfe conditions. maxiter : int, optional Maximum number of iterations to perform Returns ------- alpha_star : float or None Best alpha, or None if the line search algorithm did not converge. phi_star : float phi at alpha_star phi0 : float phi at 0 derphi_star : float or None derphi at alpha_star, or None if the line search algorithm did not converge. Notes ----- Uses the line search algorithm to enforce strong Wolfe conditions. See Wright and Nocedal, 'Numerical Optimization', 1999, pg. 59-60. For the zoom phase it uses an algorithm by [...]. gigð?g)\Âõ(ð?icS`stS(N(R'(RBR((s8lib/python2.7/site-packages/scipy/optimize/linesearch.pyt•ss7Rounding errors prevent the line search from convergings4The line search algorithm could not find a solution sless than or equal to amax: %sis*The line search algorithm did not convergeNg)\Âõ(@(R#R3RRRt_zoomtabs(RRR8R9R,RRR RFR@talpha0R:tphi_a1tphi_a0t derphi_a0RARHRIRJtmsgt derphi_a1talpha2((s8lib/python2.7/site-packages/scipy/optimize/linesearch.pyR Dst;      $               c C`s]tjddddddƒ&y |}||}||} || d|| } tjd ƒ} | d| d <|d | d <| d | d <|d| d 0 is assumed to be a descent direction. Returns ------- alpha phi1 ig@ig@ig¸…ëQ¸î?N(RR]RMR#(RR8R,RRNR!RPR:ROtfactorR`RcRTtphi_a2((s8lib/python2.7/site-packages/scipy/optimize/linesearch.pyR…›s4  #    2  (  gš™™™™™¹?gà?cC`sX|d}t|ƒ} d} d} d} xtrG|| |} || ƒ\}}|| ||| d|kr{| } Pn| d||d| d|}|| |} || ƒ\}}|| ||| d|krè| } Pn| d||d| d|}tj||| || ƒ} tj||| || ƒ} q+W| | ||fS(s> Nonmonotone backtracking line search as described in [1]_ Parameters ---------- f : callable Function returning a tuple ``(f, F)`` where ``f`` is the value of a merit function and ``F`` the residual. x_k : ndarray Initial position d : ndarray Search direction prev_fs : float List of previous merit function values. Should have ``len(prev_fs) <= M`` where ``M`` is the nonmonotonicity window parameter. eta : float Allowed merit function increase, see [1]_ gamma, tau_min, tau_max : float, optional Search parameters, see [1]_ Returns ------- alpha : float Step length xp : ndarray Next position fp : float Merit function value at next position Fp : ndarray Residual at next position References ---------- [1] "Spectral residual method without gradient information for solving large-scale nonlinear systems of equations." W. La Cruz, J.M. Martinez, M. Raydan. Math. Comp. **75**, 1429 (2006). iÿÿÿÿii(tmaxR'Rtclip(Rtx_ktdtprev_fstetatgammattau_minttau_maxtf_ktf_bartalpha_ptalpha_mRBtxptfptFptalpha_tptalpha_tm((s8lib/python2.7/site-packages/scipy/optimize/linesearch.pyt_nonmonotone_line_search_cruzÙs*(    " "!g333333ë?c C`svd} d} d} xtr1|| |}||ƒ\}}||||| d|kre| } Pn| d||d| d|}|| |}||ƒ\}}||||| d|krÒ| } Pn| d||d| d|}tj||| | | ƒ} tj||| | | ƒ} qW| |d}| |||||}|}| |||||fS(s‡ Nonmonotone line search from [1] Parameters ---------- f : callable Function returning a tuple ``(f, F)`` where ``f`` is the value of a merit function and ``F`` the residual. x_k : ndarray Initial position d : ndarray Search direction f_k : float Initial merit function value C, Q : float Control parameters. On the first iteration, give values Q=1.0, C=f_k eta : float Allowed merit function increase, see [1]_ nu, gamma, tau_min, tau_max : float, optional Search parameters, see [1]_ Returns ------- alpha : float Step length xp : ndarray Next position fp : float Merit function value at next position Fp : ndarray Residual at next position C : float New value for the control parameter C Q : float New value for the control parameter Q References ---------- .. [1] W. Cheng & D.-H. Li, ''A derivative-free nonmonotone line search and its application to the spectral residual method'', IMA J. Numer. Anal. 29, 814 (2009). ii(R'RR‹(RRŒRR“RftQRRR‘R’tnuR•R–RBR—R˜R™RšR›tQ_next((s8lib/python2.7/site-packages/scipy/optimize/linesearch.pyt_nonmonotone_line_search_cheng!s,/  " "!(((((t__doc__t __future__RRRtwarningsRtscipy.optimizeRtnumpyRtscipy._lib.sixRt__all__tRuntimeWarningRR#RR t line_searchRR RoRqRLR R‡R…RœR (((s8lib/python2.7/site-packages/scipy/optimize/linesearch.pyt sB    E P Œ "  T4 ?H