ó î&]\c@`súdZddlmZmZmZddlZddlmZm Z ddl m Z m Z m Z ddlmZddlmZdd lmZmZmZmZmZmZmZmZmZmZmZmZd „Zd „Z d „Z!d „Z"dS(s dogleg algorithm with rectangular trust regions for least-squares minimization. The description of the algorithm can be found in [Voglis]_. The algorithm does trust-region iterations, but the shape of trust regions is rectangular as opposed to conventional elliptical. The intersection of a trust region and an initial feasible region is again some rectangle. Thus on each iteration a bound-constrained quadratic optimization problem is solved. A quadratic problem is solved by well-known dogleg approach, where the function is minimized along piecewise-linear "dogleg" path [NumOpt]_, Chapter 4. If Jacobian is not rank-deficient then the function is decreasing along this path, and optimization amounts to simply following along this path as long as a point stays within the bounds. A constrained Cauchy step (along the anti-gradient) is considered for safety in rank deficient cases, in this situations the convergence might be slow. If during iterations some variable hit the initial bound and the component of anti-gradient points outside the feasible region, then a next dogleg step won't make any progress. At this state such variables satisfy first-order optimality conditions and they are excluded before computing a next dogleg step. Gauss-Newton step can be computed exactly by `numpy.linalg.lstsq` (for dense Jacobian matrices) or by iterative procedure `scipy.sparse.linalg.lsmr` (for dense and sparse matrices, or Jacobian being LinearOperator). The second option allows to solve very large problems (up to couple of millions of residuals on a regular PC), provided the Jacobian matrix is sufficiently sparse. But note that dogbox is not very good for solving problems with large number of constraints, because of variables exclusion-inclusion on each iteration (a required number of function evaluations might be high or accuracy of a solution will be poor), thus its large-scale usage is probably limited to unconstrained problems. References ---------- .. [Voglis] C. Voglis and I. E. Lagaris, "A Rectangular Trust Region Dogleg Approach for Unconstrained and Bound Constrained Nonlinear Optimization", WSEAS International Conference on Applied Mathematics, Corfu, Greece, 2004. .. [NumOpt] J. Nocedal and S. J. Wright, "Numerical optimization, 2nd edition". i(tdivisiontprint_functiontabsolute_importN(tlstsqtnorm(tLinearOperatortaslinearoperatortlsmr(tOptimizeResult(t string_typesi( tstep_size_to_boundt in_boundstupdate_tr_radiustevaluate_quadratictbuild_quadratic_1dtminimize_quadratic_1dt compute_gradtcompute_jac_scaletcheck_terminationtscale_for_robust_loss_functiontprint_header_nonlineartprint_iteration_nonlinearc`s[ˆj\}}‡‡‡fd†}‡‡‡fd†}t||fd|d|dtƒS(s¬Compute LinearOperator to use in LSMR by dogbox algorithm. `active_set` mask is used to excluded active variables from computations of matrix-vector products. c`s-|jƒjƒ}d|ˆ<ˆj|ˆƒS(Ni(traveltcopytmatvec(txtx_free(tJopt active_settd(s9lib/python2.7/site-packages/scipy/optimize/_lsq/dogbox.pyRCs c`s!ˆˆj|ƒ}d|ˆ<|S(Ni(trmatvec(Rtr(RRR(s9lib/python2.7/site-packages/scipy/optimize/_lsq/dogbox.pyRHs RRtdtype(tshapeRtfloat(RRRtmtnRR((RRRs9lib/python2.7/site-packages/scipy/optimize/_lsq/dogbox.pyt lsmr_operator;sc C`s˜||}||}tj|| ƒ}tj||ƒ}tj||ƒ}tj||ƒ} tj|| ƒ} tj||ƒ} |||| | | fS(sFind intersection of trust-region bounds and initial bounds. Returns ------- lb_total, ub_total : ndarray with shape of x Lower and upper bounds of the intersection region. orig_l, orig_u : ndarray of bool with shape of x True means that an original bound is taken as a corresponding bound in the intersection region. tr_l, tr_u : ndarray of bool with shape of x True means that a trust-region bound is taken as a corresponding bound in the intersection region. (tnptmaximumtminimumtequal( Rt tr_boundstlbtubt lb_centeredt ub_centeredtlb_totaltub_totaltorig_ltorig_uttr_lttr_u((s9lib/python2.7/site-packages/scipy/optimize/_lsq/dogbox.pytfind_intersectionPs  cC`s't||||ƒ\}} } } } } tj|dtƒ}t||| ƒr[||tfSttj|ƒ| || ƒ\}}t||d|ƒd |}||}t|||| ƒ\}}d||dk| @*s R   +