ó ‡ˆ\c@s‹dZddlZddlZddlmZmZddlmZde fd„ƒYZ d„Z d „Z dd d d e e d „ZdS(s’ Our own implementation of the Newton algorithm Unlike the scipy.optimize version, this version of the Newton conjugate gradient solver uses only one function call to retrieve the func value, the gradient value and a callable for the Hessian matvec product. If the function call is very expensive (e.g. for logistic regression with large design matrix), this approach gives very significant speedups. iÿÿÿÿN(tline_search_wolfe2tline_search_wolfe1i(tConvergenceWarningt_LineSearchErrorcBseZRS((t__name__t __module__(((s5lib/python2.7/site-packages/sklearn/utils/optimize.pyRsc Ksut||||||||}|ddkrUt||||||||}n|ddkrqtƒ‚n|S(s  Same as line_search_wolfe1, but fall back to line_search_wolfe2 if suitable step length is not found, and raise an exception if a suitable step length is not found. Raises ------ _LineSearchError If no suitable step size is found iN(RtNoneRR( tftfprimetxktpktgfktold_fvalt old_old_fvaltkwargstret((s5lib/python2.7/site-packages/sklearn/utils/optimize.pyt_line_search_wolfe12s   cCs`tjt|ƒd|jƒ}|}| }d}tj||ƒ}x||kr[tjtj|ƒƒ|krtPn||ƒ} tj|| ƒ} d| ko¼dtjtjƒj knrÅPn2| dkr÷|dkráPq÷||| |7}Pn|| } || |7}|| | }tj||ƒ} | |} | | |}|d}| }qFW|S(sI Solve iteratively the linear system 'fhess_p . xsupi = fgrad' with a conjugate gradient descent. Parameters ---------- fhess_p : callable Function that takes the gradient as a parameter and returns the matrix product of the Hessian and gradient fgrad : ndarray, shape (n_features,) or (n_features + 1,) Gradient vector maxiter : int Number of CG iterations. tol : float Stopping criterion. Returns ------- xsupi : ndarray, shape (n_features,) or (n_features + 1,) Estimated solution tdtypeiii( tnptzerostlenRtdottsumtabstfinfotfloat64teps(tfhess_ptfgradtmaxiterttoltxsupitritpsupititdri0tAptcurvtalphaitdri1tbetai((s5lib/python2.7/site-packages/sklearn/utils/optimize.pyt_cg7s4 /      g-Cëâ6?idiÈc  Cstj|ƒjƒ}|} d} |r?|||Œ} d } nx| |kr`|| |Œ\}}tj|ƒ}tj|ƒ|kr‹Pntj|ƒ}tdtj|ƒgƒ}||}t ||d|d|ƒ}d}|rEy:t ||| ||| | d|ƒ\}}}} } }WqEt k rAt j dƒPqEXn| ||} | d7} qBW| r†| |kr†t j d tƒn| | fS( sL Minimization of scalar function of one or more variables using the Newton-CG algorithm. Parameters ---------- grad_hess : callable Should return the gradient and a callable returning the matvec product of the Hessian. func : callable Should return the value of the function. grad : callable Should return the function value and the gradient. This is used by the linesearch functions. x0 : array of float Initial guess. args : tuple, optional Arguments passed to func_grad_hess, func and grad. tol : float Stopping criterion. The iteration will stop when ``max{|g_i | i = 1, ..., n} <= tol`` where ``g_i`` is the i-th component of the gradient. maxiter : int Number of Newton iterations. maxinner : int Number of CG iterations. line_search : boolean Whether to use a line search or not. warn : boolean Whether to warn when didn't converge. Returns ------- xk : ndarray of float Estimated minimum. igà?RRgð?targssLine Search failedis@newton-cg failed to converge. Increase the number of iterations.N(RtasarraytflattenRRtmaxRtmintsqrtR)RRtwarningstwarnR(t grad_hesstfunctgradtx0R*RRtmaxinnert line_searchR1R tkR R RRtabsgradtmaggradtetattermcondRtalphaktfctgctgfkp1((s5lib/python2.7/site-packages/sklearn/utils/optimize.pyt newton_cgrs:/  (    ((t__doc__tnumpyRR0tscipy.optimize.linesearchRRt exceptionsRt RuntimeErrorRRR)tTrueRA(((s5lib/python2.7/site-packages/sklearn/utils/optimize.pyt s    ;