\c @sdZddlmZddlmZddlZddlZddlZddl m Z m Z ddl m Z ddlmZd d lmZd d lmZmZmZmZd d lmZd d lmZd dlmZmZd dlmZd dlm Z ie!d6Z"e#e#ddde$ej%ej&j'e$de$e!e!d Z(deefdYZ)de)fdYZ*e!dZ+e#e$de!e$e$dej%ej&j'e!d Z,de)fdYZ-de-fd YZ.d!e*fd"YZ/dS(#st Least Angle Regression algorithm. See the documentation on the Generalized Linear Model for a complete discussion. i(tprint_function(tlogN(tlinalgt interpolate(tget_lapack_funcsi(t LinearModeli(tRegressorMixin(t arrayfuncstas_float_arrayt check_X_yt deprecated(tcheck_cv(tConvergenceWarning(tParalleltdelayed(txrange(t string_typest check_finiteiitlarcA Cs|dkr%| r%tjdtn|jd}|j}t||}| rtj|d|f}tj|d}nDtj|tj|}}tjdgtjdg}}d\}}t tj |}}tj |dtj }t }tj ||fd|j}tjd|f\}}td|f\} |dksl|t krd}|r|jd }qnt|tr|d ks|tkr|tks|jd|jdkrtj|j|}qd}n| r|j}n|dkr4tj|j|}!n |j}!| r| dkr_td qtjjd tjjntjtjj }"tjtjj!}#x trc|!jr| rtj"|!}$ntj"tj#|!}$|!|$}%| r|%}&qtj$|%}&nd}&| rg||tj%f}||}||dtj%f}||d}n|&||d<|d||#krt#|d||#kr|dkr|d||d|d}'||'|||(n||d|dkr x|;D]~}=xut8|=|D]d}1||j|1|j|1d\|j|1<|j|1d<||1d||1||1<||1d|?}@tj9|@|!f}!tj:||;}tj-|d}| dkr`td|d|>|t#|@fq`qqW| r||d }||d }| r|||j|fS|||jfSn#| r||||fS|||fSdS(s Compute Least Angle Regression or Lasso path using LARS algorithm [1] The optimization objective for the case method='lasso' is:: (1 / (2 * n_samples)) * ||y - Xw||^2_2 + alpha * ||w||_1 in the case of method='lars', the objective function is only known in the form of an implicit equation (see discussion in [1]) Read more in the :ref:`User Guide `. Parameters ----------- X : array, shape: (n_samples, n_features) Input data. y : array, shape: (n_samples) Input targets. Xy : array-like, shape (n_samples,) or (n_samples, n_targets), optional Xy = np.dot(X.T, y) that can be precomputed. It is useful only when the Gram matrix is precomputed. Gram : None, 'auto', array, shape: (n_features, n_features), optional Precomputed Gram matrix (X' * X), if ``'auto'``, the Gram matrix is precomputed from the given X, if there are more samples than features. max_iter : integer, optional (default=500) Maximum number of iterations to perform, set to infinity for no limit. alpha_min : float, optional (default=0) Minimum correlation along the path. It corresponds to the regularization parameter alpha parameter in the Lasso. method : {'lar', 'lasso'}, optional (default='lar') Specifies the returned model. Select ``'lar'`` for Least Angle Regression, ``'lasso'`` for the Lasso. copy_X : bool, optional (default=True) If ``False``, ``X`` is overwritten. eps : float, optional (default=``np.finfo(np.float).eps``) The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. copy_Gram : bool, optional (default=True) If ``False``, ``Gram`` is overwritten. verbose : int (default=0) Controls output verbosity. return_path : bool, optional (default=True) If ``return_path==True`` returns the entire path, else returns only the last point of the path. return_n_iter : bool, optional (default=False) Whether to return the number of iterations. positive : boolean (default=False) Restrict coefficients to be >= 0. This option is only allowed with method 'lasso'. Note that the model coefficients will not converge to the ordinary-least-squares solution for small values of alpha. Only coefficients up to the smallest alpha value (``alphas_[alphas_ > 0.].min()`` when fit_path=True) reached by the stepwise Lars-Lasso algorithm are typically in congruence with the solution of the coordinate descent lasso_path function. Returns -------- alphas : array, shape: [n_alphas + 1] Maximum of covariances (in absolute value) at each iteration. ``n_alphas`` is either ``max_iter``, ``n_features`` or the number of nodes in the path with ``alpha >= alpha_min``, whichever is smaller. active : array, shape [n_alphas] Indices of active variables at the end of the path. coefs : array, shape (n_features, n_alphas + 1) Coefficients along the path n_iter : int Number of iterations run. Returned only if return_n_iter is set to True. See also -------- lasso_path LassoLars Lars LassoLarsCV LarsCV sklearn.decomposition.sparse_encode References ---------- .. [1] "Least Angle Regression", Effron et al. http://statweb.stanford.edu/~tibs/ftp/lars.pdf .. [2] `Wikipedia entry on the Least-angle regression `_ .. [3] `Wikipedia entry on the Lasso `_ Rs|positive option is broken for Least Angle Regression (LAR). Use method="lasso". This option will be removed in version 0.22.igitdtypetswaptnrm2tpotrstFtautos(Step Added Dropped Active set size Ct.iNttranstlowert overwrite_bgHz>sRegressors in active set degenerate. Dropping a regressor, after %i iterations, i.e. alpha=%.3e, with an active set of %i regressors, and the smallest cholesky pivot element being %.3e. Reduce max_iter or increase eps parameters.s%s %s %s %s %sittlassosEarly stopping the lars path, as the residues are small and the current value of alpha is no longer well controlled. %i iterations, alpha=%.3e, previous alpha=%.3e, with an active set of %i regressors..g?(ii(RR(R(;twarningstwarntDeprecationWarningtshapetsizetmintnptzerostarraytlisttarangetemptytint8tFalseRRtget_blas_funcsRtNonetcopyt isinstanceRtTruetdottTtprinttsyststdouttwritetflushtfinfotfloat32ttinytepstargmaxtabstfabstnewaxist ones_liketsigntsolve_triangulartsolve_triangular_argstmaxtsqrtR tappendtsumtisfinitetflatRtmin_postwheretresizet zeros_liketcholesky_deletetpoptrangetr_tdelete(AtXtytXytGramtmax_itert alpha_mintmethodtcopy_XR<t copy_Gramtverboset return_patht return_n_itertpositivet n_featurest n_samplest max_featurestcoefstalphastcoeft prev_coeftalphat prev_alphatn_itertn_activetactivetindicest sign_activetdroptLRRtsolve_choleskytCovttiny32tequality_tolerancetC_idxtC_tCtsstmtntCov_not_shortenedtctvtdiagt least_squarestinfotAAtitL_ttmpteq_dirt corr_eq_dirtg1tgamma_tg2tztz_postidxt add_featurestiitdrop_idxtresidualttemp((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyt lars_path!sr    % '&               %  16%U&  2%  %  ,    $"(   "  &       6  "  9-0 %-O  tLarsc BsneZdZdZeeeddejejj eeed Z e dZ ddZddZRS( sLeast Angle Regression model a.k.a. LAR Read more in the :ref:`User Guide `. Parameters ---------- fit_intercept : boolean Whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). verbose : boolean or integer, optional Sets the verbosity amount normalize : boolean, optional, default True This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. precompute : True | False | 'auto' | array-like Whether to use a precomputed Gram matrix to speed up calculations. If set to ``'auto'`` let us decide. The Gram matrix can also be passed as argument. n_nonzero_coefs : int, optional Target number of non-zero coefficients. Use ``np.inf`` for no limit. eps : float, optional The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. Unlike the ``tol`` parameter in some iterative optimization-based algorithms, this parameter does not control the tolerance of the optimization. copy_X : boolean, optional, default True If ``True``, X will be copied; else, it may be overwritten. fit_path : boolean If True the full path is stored in the ``coef_path_`` attribute. If you compute the solution for a large problem or many targets, setting ``fit_path`` to ``False`` will lead to a speedup, especially with a small alpha. positive : boolean (default=False) Restrict coefficients to be >= 0. Be aware that you might want to remove fit_intercept which is set True by default. .. deprecated:: 0.20 The option is broken and deprecated. It will be removed in v0.22. Attributes ---------- alphas_ : array, shape (n_alphas + 1,) | list of n_targets such arrays Maximum of covariances (in absolute value) at each iteration. ``n_alphas`` is either ``n_nonzero_coefs`` or ``n_features``, whichever is smaller. active_ : list, length = n_alphas | list of n_targets such lists Indices of active variables at the end of the path. coef_path_ : array, shape (n_features, n_alphas + 1) | list of n_targets such arrays The varying values of the coefficients along the path. It is not present if the ``fit_path`` parameter is ``False``. coef_ : array, shape (n_features,) or (n_targets, n_features) Parameter vector (w in the formulation formula). intercept_ : float | array, shape (n_targets,) Independent term in decision function. n_iter_ : array-like or int The number of iterations taken by lars_path to find the grid of alphas for each target. Examples -------- >>> from sklearn import linear_model >>> reg = linear_model.Lars(n_nonzero_coefs=1) >>> reg.fit([[-1, 1], [0, 0], [1, 1]], [-1.1111, 0, -1.1111]) ... # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE Lars(copy_X=True, eps=..., fit_intercept=True, fit_path=True, n_nonzero_coefs=1, normalize=True, positive=False, precompute='auto', verbose=False) >>> print(reg.coef_) # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE [ 0. -1.11...] See also -------- lars_path, LarsCV sklearn.decomposition.sparse_encode RRic CsU||_||_||_||_||_| |_||_||_||_dS(N( t fit_interceptR]t normalizet precomputetn_nonzero_coefsR`R<R[tfit_path( tselfRR]RRRR<R[RR`((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyt__init__Ss        cCs}t|d ry|tksa|dkrB|jd|jdksa|dkry|jddkrytj|j|}n|S(Nt __array__Rii(thasattrR1R"R%R2R3(RRTRU((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyt _get_gramas  &cCs|jd}|j|||j|j|j\}}}} } |jdkrk|ddtjf}n|jd} |j|j ||} g|_ g|_ tj | |f|_ |rlg|_g|_x(t| D]} |dkrdn|dd| f}t||dd| fd| d|d|jdtd|d|jd td |jdd |d |jd tdtd|j \}}}}|j j||jj||j j||jj||dddf|j | `. Parameters ---------- alpha : float Constant that multiplies the penalty term. Defaults to 1.0. ``alpha = 0`` is equivalent to an ordinary least square, solved by :class:`LinearRegression`. For numerical reasons, using ``alpha = 0`` with the LassoLars object is not advised and you should prefer the LinearRegression object. fit_intercept : boolean whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). verbose : boolean or integer, optional Sets the verbosity amount normalize : boolean, optional, default True This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. precompute : True | False | 'auto' | array-like Whether to use a precomputed Gram matrix to speed up calculations. If set to ``'auto'`` let us decide. The Gram matrix can also be passed as argument. max_iter : integer, optional Maximum number of iterations to perform. eps : float, optional The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. Unlike the ``tol`` parameter in some iterative optimization-based algorithms, this parameter does not control the tolerance of the optimization. copy_X : boolean, optional, default True If True, X will be copied; else, it may be overwritten. fit_path : boolean If ``True`` the full path is stored in the ``coef_path_`` attribute. If you compute the solution for a large problem or many targets, setting ``fit_path`` to ``False`` will lead to a speedup, especially with a small alpha. positive : boolean (default=False) Restrict coefficients to be >= 0. Be aware that you might want to remove fit_intercept which is set True by default. Under the positive restriction the model coefficients will not converge to the ordinary-least-squares solution for small values of alpha. Only coefficients up to the smallest alpha value (``alphas_[alphas_ > 0.].min()`` when fit_path=True) reached by the stepwise Lars-Lasso algorithm are typically in congruence with the solution of the coordinate descent Lasso estimator. Attributes ---------- alphas_ : array, shape (n_alphas + 1,) | list of n_targets such arrays Maximum of covariances (in absolute value) at each iteration. ``n_alphas`` is either ``max_iter``, ``n_features``, or the number of nodes in the path with correlation greater than ``alpha``, whichever is smaller. active_ : list, length = n_alphas | list of n_targets such lists Indices of active variables at the end of the path. coef_path_ : array, shape (n_features, n_alphas + 1) or list If a list is passed it's expected to be one of n_targets such arrays. The varying values of the coefficients along the path. It is not present if the ``fit_path`` parameter is ``False``. coef_ : array, shape (n_features,) or (n_targets, n_features) Parameter vector (w in the formulation formula). intercept_ : float | array, shape (n_targets,) Independent term in decision function. n_iter_ : array-like or int. The number of iterations taken by lars_path to find the grid of alphas for each target. Examples -------- >>> from sklearn import linear_model >>> reg = linear_model.LassoLars(alpha=0.01) >>> reg.fit([[-1, 1], [0, 0], [1, 1]], [-1, 0, -1]) ... # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE LassoLars(alpha=0.01, copy_X=True, eps=..., fit_intercept=True, fit_path=True, max_iter=500, normalize=True, positive=False, precompute='auto', verbose=False) >>> print(reg.coef_) # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE [ 0. -0.963257...] See also -------- lars_path lasso_path Lasso LassoCV LassoLarsCV LassoLarsIC sklearn.decomposition.sparse_encode Rg?Ric Cs^||_||_||_||_||_| |_||_||_||_| |_ dS(N( RhRRXR]RR`RR[R<R( RRhRR]RRRXR<R[RR`((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyRBs         ( RRRRZR1R,R%R9RR<R(((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyRs v  cCs!|s|jj r|jS|S(N(tflagst writeableR/(R'R/((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyt_check_copy_and_writeableUs tlarsc Cst||}t||}t||}t||}|r|jdd} || 8}|| 8}|jdd}t|dt}||8}t|dt}||8}n| rtjtj|ddd}tj|}|dd|fc||:= 0. Be aware that you might want to remove fit_intercept which is set True by default. See reservations for using this option in combination with method 'lasso' for expected small values of alpha in the doc of LassoLarsCV and LassoLarsIC. normalize : boolean, optional, default True This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. max_iter : integer, optional Maximum number of iterations to perform. eps : float, optional The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. Unlike the ``tol`` parameter in some iterative optimization-based algorithms, this parameter does not control the tolerance of the optimization. Returns -------- alphas : array, shape (n_alphas,) Maximum of covariances (in absolute value) at each iteration. ``n_alphas`` is either ``max_iter`` or ``n_features``, whichever is smaller. active : list Indices of active variables at the end of the path. coefs : array, shape (n_features, n_alphas) Coefficients along the path residues : array, shape (n_alphas, n_samples) Residues of the prediction on the test data taxisiR/iNRWR[R\RZR]iRXR<R`( RtmeanRR,R%RFRHt flatnonzeroRRER@R2R3(tX_trainty_traintX_testty_testRWR/RZR]RRRXR<R`tX_meanty_meantnormstnonzerosReRlRdtresidues((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyt_lars_path_residues[s2S    "#%*)tLarsCVc BsqeZdZdZeededddd ejej j eed Z dZ e edd ZRS( sCross-validated Least Angle Regression model. See glossary entry for :term:`cross-validation estimator`. Read more in the :ref:`User Guide `. Parameters ---------- fit_intercept : boolean whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). verbose : boolean or integer, optional Sets the verbosity amount max_iter : integer, optional Maximum number of iterations to perform. normalize : boolean, optional, default True This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. precompute : True | False | 'auto' | array-like Whether to use a precomputed Gram matrix to speed up calculations. If set to ``'auto'`` let us decide. The Gram matrix cannot be passed as argument since we will use only subsets of X. cv : int, cross-validation generator or an iterable, optional Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 3-fold cross-validation, - integer, to specify the number of folds. - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. For integer/None inputs, :class:`KFold` is used. Refer :ref:`User Guide ` for the various cross-validation strategies that can be used here. .. versionchanged:: 0.20 ``cv`` default value if None will change from 3-fold to 5-fold in v0.22. max_n_alphas : integer, optional The maximum number of points on the path used to compute the residuals in the cross-validation n_jobs : int or None, optional (default=None) Number of CPUs to use during the cross validation. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. eps : float, optional The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. copy_X : boolean, optional, default True If ``True``, X will be copied; else, it may be overwritten. positive : boolean (default=False) Restrict coefficients to be >= 0. Be aware that you might want to remove fit_intercept which is set True by default. .. deprecated:: 0.20 The option is broken and deprecated. It will be removed in v0.22. Attributes ---------- coef_ : array, shape (n_features,) parameter vector (w in the formulation formula) intercept_ : float independent term in decision function coef_path_ : array, shape (n_features, n_alphas) the varying values of the coefficients along the path alpha_ : float the estimated regularization parameter alpha alphas_ : array, shape (n_alphas,) the different values of alpha along the path cv_alphas_ : array, shape (n_cv_alphas,) all the values of alpha along the path for the different folds mse_path_ : array, shape (n_folds, n_cv_alphas) the mean square error on left-out for each fold along the path (alpha values given by ``cv_alphas``) n_iter_ : array-like or int the number of iterations run by Lars with the optimal alpha. Examples -------- >>> from sklearn.linear_model import LarsCV >>> from sklearn.datasets import make_regression >>> X, y = make_regression(n_samples=200, noise=4.0, random_state=0) >>> reg = LarsCV(cv=5).fit(X, y) >>> reg.score(X, y) # doctest: +ELLIPSIS 0.9996... >>> reg.alpha_ 0.0254... >>> reg.predict(X[:1,]) array([154.0842...]) See also -------- lars_path, LassoLars, LassoLarsCV RiRR ic Csq||_||_||_||_tt|jd|d|d|d|ddd| d| d td | dS( NRR]RRRiR<R[RR`(RXtcvt max_n_alphastn_jobstsuperRRR1( RRR]RXRRRRRR<R[R`((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyRGs     c s9tdt\tdjtdjtjdt}jtdrt j dj j dnt djdjfd |jD}tjtt|d }tj|}ttd tt|tj}|d d |}tjt|t|f}x-t|D]\}\} } } } | d d d } | d d d } | d d krtjd | f} tj| d tjf| f} n| d |d krGtj| |d f} tj| | d tjff} ntj | | dd |} | dC} tj!| dd |d d |fvsiiNiRiRXRhRVR(+R R1RR[R RR,RRRR t __class__RR RR]tsplitR%t concatenateR(tziptuniquetintREtlenRRR*t enumerateRRR@Rtinterp1dRtallRItargmintalpha_t cv_alphas_t mse_path_RRXR.(RRTRURtcv_pathst all_alphaststridetmse_pathtindexReRlRdRt this_residuestmaskt i_best_alphat best_alpha((RWRTRRUs?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyRVsT   .!%##  )       sYAttribute alpha is deprecated in 0.19 and will be removed in 0.21. See ``alpha_`` insteadcCs|jS(N(R(R((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyRhsN(RRRRZR1R,R.R%R9RR<RRtpropertyR Rh(((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyRsw   Pt LassoLarsCVc BsMeZdZdZeededdddejej j eed Z RS(sCross-validated Lasso, using the LARS algorithm. See glossary entry for :term:`cross-validation estimator`. The optimization objective for Lasso is:: (1 / (2 * n_samples)) * ||y - Xw||^2_2 + alpha * ||w||_1 Read more in the :ref:`User Guide `. Parameters ---------- fit_intercept : boolean whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). verbose : boolean or integer, optional Sets the verbosity amount max_iter : integer, optional Maximum number of iterations to perform. normalize : boolean, optional, default True This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. precompute : True | False | 'auto' Whether to use a precomputed Gram matrix to speed up calculations. If set to ``'auto'`` let us decide. The Gram matrix cannot be passed as argument since we will use only subsets of X. cv : int, cross-validation generator or an iterable, optional Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 3-fold cross-validation, - integer, to specify the number of folds. - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. For integer/None inputs, :class:`KFold` is used. Refer :ref:`User Guide ` for the various cross-validation strategies that can be used here. .. versionchanged:: 0.20 ``cv`` default value if None will change from 3-fold to 5-fold in v0.22. max_n_alphas : integer, optional The maximum number of points on the path used to compute the residuals in the cross-validation n_jobs : int or None, optional (default=None) Number of CPUs to use during the cross validation. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. eps : float, optional The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. copy_X : boolean, optional, default True If True, X will be copied; else, it may be overwritten. positive : boolean (default=False) Restrict coefficients to be >= 0. Be aware that you might want to remove fit_intercept which is set True by default. Under the positive restriction the model coefficients do not converge to the ordinary-least-squares solution for small values of alpha. Only coefficients up to the smallest alpha value (``alphas_[alphas_ > 0.].min()`` when fit_path=True) reached by the stepwise Lars-Lasso algorithm are typically in congruence with the solution of the coordinate descent Lasso estimator. As a consequence using LassoLarsCV only makes sense for problems where a sparse solution is expected and/or reached. Attributes ---------- coef_ : array, shape (n_features,) parameter vector (w in the formulation formula) intercept_ : float independent term in decision function. coef_path_ : array, shape (n_features, n_alphas) the varying values of the coefficients along the path alpha_ : float the estimated regularization parameter alpha alphas_ : array, shape (n_alphas,) the different values of alpha along the path cv_alphas_ : array, shape (n_cv_alphas,) all the values of alpha along the path for the different folds mse_path_ : array, shape (n_folds, n_cv_alphas) the mean square error on left-out for each fold along the path (alpha values given by ``cv_alphas``) n_iter_ : array-like or int the number of iterations run by Lars with the optimal alpha. Examples -------- >>> from sklearn.linear_model import LassoLarsCV >>> from sklearn.datasets import make_regression >>> X, y = make_regression(noise=4.0, random_state=0) >>> reg = LassoLarsCV(cv=5).fit(X, y) >>> reg.score(X, y) # doctest: +ELLIPSIS 0.9992... >>> reg.alpha_ 0.0484... >>> reg.predict(X[:1,]) array([-77.8723...]) Notes ----- The object solves the same problem as the LassoCV object. However, unlike the LassoCV, it find the relevant alphas values by itself. In general, because of this property, it will be more stable. However, it is more fragile to heavily multicollinear datasets. It is more efficient than the LassoCV if only a small number of features are selected compared to the total number, for instance if there are very few samples compared to the number of features. See also -------- lars_path, LassoLars, LarsCV, LassoCV RiRR ic Csg||_||_||_||_||_||_||_||_| |_| |_ | |_ dS(N( RR]RXRRRRRR<R[R`( RRR]RXRRRRRR<R[R`((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyR>s          N( RRRRZR1R,R.R%R9RR<R(((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyRs   t LassoLarsICc BsMeZdZdeeeddejejjeed Z edZ RS(sLasso model fit with Lars using BIC or AIC for model selection The optimization objective for Lasso is:: (1 / (2 * n_samples)) * ||y - Xw||^2_2 + alpha * ||w||_1 AIC is the Akaike information criterion and BIC is the Bayes Information criterion. Such criteria are useful to select the value of the regularization parameter by making a trade-off between the goodness of fit and the complexity of the model. A good model should explain well the data while being simple. Read more in the :ref:`User Guide `. Parameters ---------- criterion : 'bic' | 'aic' The type of criterion to use. fit_intercept : boolean whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). verbose : boolean or integer, optional Sets the verbosity amount normalize : boolean, optional, default True This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. precompute : True | False | 'auto' | array-like Whether to use a precomputed Gram matrix to speed up calculations. If set to ``'auto'`` let us decide. The Gram matrix can also be passed as argument. max_iter : integer, optional Maximum number of iterations to perform. Can be used for early stopping. eps : float, optional The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. Unlike the ``tol`` parameter in some iterative optimization-based algorithms, this parameter does not control the tolerance of the optimization. copy_X : boolean, optional, default True If True, X will be copied; else, it may be overwritten. positive : boolean (default=False) Restrict coefficients to be >= 0. Be aware that you might want to remove fit_intercept which is set True by default. Under the positive restriction the model coefficients do not converge to the ordinary-least-squares solution for small values of alpha. Only coefficients up to the smallest alpha value (``alphas_[alphas_ > 0.].min()`` when fit_path=True) reached by the stepwise Lars-Lasso algorithm are typically in congruence with the solution of the coordinate descent Lasso estimator. As a consequence using LassoLarsIC only makes sense for problems where a sparse solution is expected and/or reached. Attributes ---------- coef_ : array, shape (n_features,) parameter vector (w in the formulation formula) intercept_ : float independent term in decision function. alpha_ : float the alpha parameter chosen by the information criterion n_iter_ : int number of iterations run by lars_path to find the grid of alphas. criterion_ : array, shape (n_alphas,) The value of the information criteria ('aic', 'bic') across all alphas. The alpha which has the smallest information criterion is chosen. This value is larger by a factor of ``n_samples`` compared to Eqns. 2.15 and 2.16 in (Zou et al, 2007). Examples -------- >>> from sklearn import linear_model >>> reg = linear_model.LassoLarsIC(criterion='bic') >>> reg.fit([[-1, 1], [0, 0], [1, 1]], [-1.1111, 0, -1.1111]) ... # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE LassoLarsIC(copy_X=True, criterion='bic', eps=..., fit_intercept=True, max_iter=500, normalize=True, positive=False, precompute='auto', verbose=False) >>> print(reg.coef_) # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE [ 0. -1.11...] Notes ----- The estimation of the number of degrees of freedom is given by: "On the degrees of freedom of the lasso" Hui Zou, Trevor Hastie, and Robert Tibshirani Ann. Statist. Volume 35, Number 5 (2007), 2173-2192. https://en.wikipedia.org/wiki/Akaike_information_criterion https://en.wikipedia.org/wiki/Bayesian_information_criterion See also -------- lars_path, LassoLars, LassoLarsCV taicRic Cs^||_||_| |_||_||_||_||_||_||_t |_ dS(N( t criterionRR`RXR]RR[RR<R1R( RRRR]RRRXR<R[R`((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyRs         cCsut||dt\}}tj|||j|j|j\}}}}}|j}|j}t ||d|d|dtddddd |j d |d |j d td |j  \} } } |_ |jd} |jdkrd} n*|jdkrt| } n td|ddtjftj|| }tj|ddd}tj|}tj| jddtj}xit| jD]X\}}tj|tj|jj k}tj|sqntj|||R9RtanyRHRt criterion_RRRR(RRTRUR[tXmeantymeantXstdRXRWRRRRbtKtRtmean_squared_errortsigma2tdfRRfRteps64tn_best((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyRsB-  !*   )$  ( RRRR1R,R%R9RR<RR(((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyRQs t  ! (0Rt __future__RtmathRR5RtnumpyR%tscipyRRtscipy.linalg.lapackRtbaseRRtutilsRRR R tmodel_selectionR t exceptionsR t utils._joblibR Rtexternals.six.movesRt externals.sixRR,RDR.R1R9RR<RRRRRRRR(((s?lib/python2.7/site-packages/sklearn/linear_model/least_angle.pyts@   "       n