B lZ1 @sddlZddlmZddlmmZddlm Z ddZ dddZ dddZ GdddeZ GdddejZeee dS)N)Results)cache_readonlyc s@fdd}fdd}fdd}|||fS)ao Negative penalized log-likelihood functions. Returns the negative penalized log-likelihood, its derivative, and its Hessian. The penalty only includes the smooth (L2) term. All three functions have argument signature (x, model), where ``x`` is a point in the parameter space and ``model`` is an arbitrary statsmodels regression model. csJ|j}dt|dd}|jtj|f}| ||S)N)nobsnpsumZlogliker_)paramsmodelrZpen_llfZllf)L1_wtalphak loglike_kwds;lib/python3.7/site-packages/statsmodels/base/elastic_net.py nploglike)s"z_gen_npfuncs..nploglikecs@|j}d|}|jtj|fd |}||S)Nrr)rZscorerr )r r rZpen_gradZgr)r r r score_kwdsrrnpscore/sz_gen_npfuncs..npscorecs<|j}d}|jtj|fd ||}|S)Nr)rr)rZhessianrr )r r rZpen_hessh)r r hess_kwdsrrrnphess5s"z_gen_npfuncs..nphessr) rr r rrrrrrr)r r rrrrr _gen_npfuncss r coord_descentd?Hz>:0yE>FTc ' sjjd} dkrindkr(indkr8intrTt| |dkrht| }n|}d}tjt|td} }d|d<| dd}d|kr|ddk r|dkrt | d}n|t | d7}fd d t | D}xt |D] }|}xt | D]}||r8q&|}d ||<t j|}|dk rh||7}jjjdd|ffd|i|}||\}}}t||||||||| d ||<|d kr&t|||kr&d ||<d||<q&Wtt||}||krPqWd |t||k<|sJt|}t|St|}t| | f} tfdd jD}t|d krΈjjjdd|ff|}!|!}"|"j||<|"j| t||<n,jjjddd ff|}!|!jd d}"t|"jtjr|"jj}#n|"j}#t |"dr.|"j!}$nd}$j"j#}%}&t|_"j$j"_#|#|| |$d}d |_%||_&d|di|_'|%|&_"_#|S)ay Return an elastic net regularized fit to a regression model. Parameters ---------- model : model object A statsmodels object implementing ``loglike``, ``score``, and ``hessian``. method : Only the coordinate descent algorithm is implemented. maxiter : integer The maximum number of iteration cycles (an iteration cycle involves running coordinate descent on all variables). alpha : scalar or array-like The penalty weight. If a scalar, the same penalty weight applies to all variables in the model. If a vector, it must have the same length as `params`, and contains a penalty weight for each coefficient. L1_wt : scalar The fraction of the penalty given to the L1 penalty term. Must be between 0 and 1 (inclusive). If 0, the fit is a ridge fit, if 1 it is a lasso fit. start_params : array-like Starting values for `params`. cnvrg_tol : scalar If `params` changes by less than this amount (in sup-norm) in one iteration cycle, the algorithm terminates with convergence. zero_tol : scalar Any estimated coefficient smaller than this value is replaced with zero. refit : bool If True, the model is refit using only the variables that have non-zero coefficients in the regularized fit. The refitted model is not regularized. check_step : bool If True, confirm that the first step is an improvement and search further if it is not. loglike_kwds : dict-like or None Keyword arguments for the log-likelihood function. score_kwds : dict-like or None Keyword arguments for the score function. hess_kwds : dict-like or None Keyword arguments for the Hessian function. Returns ------- A results object. Notes ----- The ``elastic net`` penalty is a combination of L1 and L2 penalties. The function that is minimized is: -loglike/n + alpha*((1-L1_wt)*|params|_2^2/2 + L1_wt*|params|_1) where |*|_1 and |*|_2 are the L1 and L2 norms. The computational approach used here is to obtain a quadratic approximation to the smooth part of the target function: -loglike/n + alpha*(1-L1_wt)*|params|_2^2/2 then repeatedly optimize the L1 penalized version of this function along coordinate axes. rNg-C6?)ZdtypeFZhasconstoffsetZexposurec sg|]}t|qSr)r).0r)r r rrrrr sz"fit_elasticnet..r)tol check_stepTgcsg|]}|t|dfqS)N)getattr)r r)r rrr!s)maxiterscaleg?)r&Z iteration)(ZexogshaperZisscalarZonesZzeroscopylenboolZ_get_init_kwdspoplograngedot __class__Zendog_opt_1dabsmaxRegularizedResultsRegularizedResultsWrapperZ flatnonzerodictZ _init_keysZfitr Znormalized_cov_paramsZix_ issubclasswrapResultsWrapperZ_resultshasattrr&Zdf_modelZdf_residrZ regularizedmethodZ fit_history)'r r:r%r r Z start_paramsZ cnvrg_tolZzero_tolZrefitr#rrrZk_exogr ZbtolZ params_zeroZ init_argsZ model_offsetZfgh_listZitrZ params_saverZparams0rZ model_1varfuncgradhessZpchangeZresultsZiiZcovZmodel1Zrsltklassr&pqr)r r rrr rrfit_elasticnet>sI       $         rAcCs|}|||} |||} |||} | | |} |t| kr@dS| dkrV|| | } n| dkrn||  | } ntjS|s|| S||| ||t|| }|| |t|dkr|| Sddlm}|||f|d|df|d}|S)az One-dimensional helper for elastic net. Parameters ---------- func : function A smooth function of a single variable to be optimized with L1 penaty. grad : function The gradient of `func`. hess : function The Hessian of `func`. model : statsmodels model The model being fit. start : real A starting value for the function argument L1_wt : non-negative real The weight for the L1 penalty function. tol : non-negative real A convergence threshold. check_step : bool If True, check that the first step is an improvement and use bisection if it is not. If False, return after the first step regardless. Notes ----- ``func``, ``grad``, and ``hess`` have argument signature (x, model), where ``x`` is a point in the parameter space and ``model`` is the model being fit. If the log-likelihood for the model is exactly quadratic, the global minimum is returned in one step. Otherwise numerical bisection is used. Returns ------- The argmin of the objective function. grg|=)brentr)argsZbrackr")rr1nanZscipy.optimizerB)r;r<r=r startr r"r#xfbcdrf1rBZx_optrrrr0 s(5      r0cs(eZdZfddZeddZZS)r3cstt|||dS)N)superr3__init__)selfr r )r/rrrMbszRegularizedResults.__init__cCs|j|jS)N)r Zpredictr )rNrrr fittedvalueseszRegularizedResults.fittedvalues)__name__ __module__ __qualname__rMrrO __classcell__rr)r/rr3`s r3c@seZdZddddZeZdS)r4columnsZrows)r ZresidrON)rPrQrRZ_attrsZ _wrap_attrsrrrrr4jsr4) rrrrNrrFTNNN)T)ZnumpyrZstatsmodels.base.modelrZstatsmodels.base.wrapperbasewrapperr7Zstatsmodels.tools.decoratorsrrrAr0r3r8r4Zpopulate_wrapperrrrrs  ! K U