B \d@sdZddlmZmZddlZddlZddlmZddlm Z ddl mZ ddl m Z mZmZddlmZd d l mZd d lmZd d lmZd d lmZd dlmZd dlmZd dlmZd dlmZd dlmZd dlmZd dl m!Z!d dl"m#Z#d dl$m%Z%d4ddZ&d5ddZ'ddZ(d6dd Z)d!d"Z*d7d$d%Z+Gd&d'd'e!,eeZ-Gd(d)d)e-eZ.Gd*d+d+e e-Z/Gd,d-d-eZ0Gd.d/d/eZ1Gd0d1d1e1eZ2Gd2d3d3e e1Z3dS)8z Ridge regression )ABCMetaabstractmethodN)linalg)sparse)LinearClassifierMixin LinearModel _rescale_data) sag_solver)RegressorMixin)safe_sparse_dot) row_norms) check_X_y) check_array)check_consistent_length)compute_sample_weight) column_or_1d)LabelBinarizer) GridSearchCV)six) check_scoring)ConvergenceWarningMbP?c s|j\}}t|tj|jd|f|jd}||krDfdd} n fdd} xRt|jdD]>} |dd| f} | || } ||krtj||f| |jd} ytj| | |dd\}}Wn(t k rtj| | |d \}}YnX ||| <nr | } tj||f| |jd} y tj| | ||dd \|| <}Wn0t k rftj| | ||d \|| <}YnX|d kr~t d ||dkrb|d krb|rbt d|tqbW|S)Nr)dtypecsfdd}|S)Ncs||S)N)matvecrmatvec)x)X1 curr_alpha9lib/python3.7/site-packages/sklearn/linear_model/ridge.py_mv,sz0_solve_sparse_cg..create_mv.._mvr )rr")r)rr! create_mv+sz#_solve_sparse_cg..create_mvcsfdd}|S)Ncs||S)N)rr)r)rrr r!r"1sz0_solve_sparse_cg..create_mv.._mvr )rr")r)rr!r#0s)rrZlegacy)tolatol)r$)maxiterr$r%)r&r$rzFailed with error code %dz/sparse_cg did not converge after %d iterations.)shape sp_linalgZaslinearoperatornpemptyrrangeZLinearOperatorZcg TypeErrorr ValueErrorwarningswarnr)Xyalphamax_iterr$verbose n_samples n_featurescoefsr#iy_columnZmvCcoefinfor )rr!_solve_sparse_cg%s@         r=c Cs|j\}}tj|jd|f|jd}tj|jdtjd}t|} xXt|jdD]F} |dd| f} tj|| | | |||d} | d|| <| d|| <qTW||fS)Nr)r)Zdampr%ZbtolZiter_limrr ) r'r)r*rint32sqrtr+r(lsqr) r0r1r2r3r$r5r6r7n_iterZ sqrt_alphar8r9r<r r r! _solve_lsqr^s    rBc Cs|j\}}|jd}t|j|dd}t|j|dd}t|t||dg}|r|jdd|d|d7<tj||dddjStj ||g|j d} xrt | |j|D]`\} } } |jdd|d| 7<tj|| ddd | dd<|jdd|d| 8<qW| SdS)NrT) dense_outputr)sym_pos overwrite_a)rF) r'r Tr)Z array_equallenflatrsolver*rzipravel) r0r1r2r5r6 n_targetsAZXy one_alphar7r;target current_alphar r r!_solve_choleskyps        rQFcCs|jd}|jd}|r |}t|}||dk}t|tjpL|dk}|rtt|} || ddtjf}|t | | 9}|r8|j dd|d|d7<yt j ||ddd} Wn2tj j k rtdt ||d} YnX|j dd|d|d8<|r4| | ddtjf9} | St||g|j} xtt| |j|D]b\} } } |j dd|d| 7<t j || ddd| dd<|j dd|d| 8<qZW|r| | tjddf9} | jSdS)Nrr)g?NTF)rDrEzNSingular matrix in solving dual problem. Using least-squares solution instead.)r'copyr) atleast_1dall isinstanceZndarrayr?newaxisZouterrHrrI LinAlgErrorr.r/Zlstsqr*rrJrFrK)Kr1r2 sample_weightrRr5rLrNhas_swsw dual_coefZ dual_coefsrOrPr r r!_solve_cholesky_kernelsB          "r]c Cstj|dd\}}}|dk}||ddtjf}t|j|}tj|j|jf|jd} ||d|| |<| |} t|j| jS)NF) full_matricesgV瞯<)rr ) rsvdr)rVdotrFzerossizer) r0r1r2UsZVtidxZs_nnzZUTydZd_UT_yr r r! _solve_svdsrgautoc Cs>| r,t|r,|dkr,|dkr(tdd}tjtjg} |dkrht|dgtjdd}t|tjdd d }n$t|dd d g| d }t||jdd}t |||j \} } |j dkrt dt |j d}|j dkr|dd}d}|j \}}| |krt d| |f|dk }|dkr*t|r |r&d}nd}|rdt|j dkrJt d|dkrdt|||\}}tj||jd}|jd|gkrt d|j|f|jdkr|dkrt||}|dkrt d|d}|dkrt||||||}n|dkrt|||||\}}n|dkr| | kr~t||jdd}y"t|||}t|j|ddj}Wntjk rzd}YnXn.yt|||}Wntjk rd}YnXn|dkrt|dd }t|j d| f}tj|j dtj d}t!|j df}xt"t#||jD]\}\}}d!t!| t$| dfi}t%|||d"|d#||||d|||d$kd%\}}}| r|dd||<|d||<n|||<|||<qW|j d#dkr|d#}t|}|dkrt|rt&d&t'|||}|r|}| r| r|||fS| r(||fS| r6||fS|SdS)'aSolve the ridge equation by the method of normal equations. Read more in the :ref:`User Guide `. Parameters ---------- X : {array-like, sparse matrix, LinearOperator}, shape = [n_samples, n_features] Training data y : array-like, shape = [n_samples] or [n_samples, n_targets] Target values alpha : {float, array-like}, shape = [n_targets] if array-like Regularization strength; must be a positive float. Regularization improves the conditioning of the problem and reduces the variance of the estimates. Larger values specify stronger regularization. Alpha corresponds to ``C^-1`` in other linear models such as LogisticRegression or LinearSVC. If an array is passed, penalties are assumed to be specific to the targets. Hence they must correspond in number. sample_weight : float or numpy array of shape [n_samples] Individual weights for each sample. If sample_weight is not None and solver='auto', the solver will be set to 'cholesky'. .. versionadded:: 0.17 solver : {'auto', 'svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag', 'saga'} Solver to use in the computational routines: - 'auto' chooses the solver automatically based on the type of data. - 'svd' uses a Singular Value Decomposition of X to compute the Ridge coefficients. More stable for singular matrices than 'cholesky'. - 'cholesky' uses the standard scipy.linalg.solve function to obtain a closed-form solution via a Cholesky decomposition of dot(X.T, X) - 'sparse_cg' uses the conjugate gradient solver as found in scipy.sparse.linalg.cg. As an iterative algorithm, this solver is more appropriate than 'cholesky' for large-scale data (possibility to set `tol` and `max_iter`). - 'lsqr' uses the dedicated regularized least-squares routine scipy.sparse.linalg.lsqr. It is the fastest and uses an iterative procedure. - 'sag' uses a Stochastic Average Gradient descent, and 'saga' uses its improved, unbiased version named SAGA. Both methods also use an iterative procedure, and are often faster than other solvers when both n_samples and n_features are large. Note that 'sag' and 'saga' fast convergence is only guaranteed on features with approximately the same scale. You can preprocess the data with a scaler from sklearn.preprocessing. All last five solvers support both dense and sparse data. However, only 'sag' and 'saga' supports sparse input when`fit_intercept` is True. .. versionadded:: 0.17 Stochastic Average Gradient descent solver. .. versionadded:: 0.19 SAGA solver. max_iter : int, optional Maximum number of iterations for conjugate gradient solver. For the 'sparse_cg' and 'lsqr' solvers, the default value is determined by scipy.sparse.linalg. For 'sag' and saga solver, the default value is 1000. tol : float Precision of the solution. verbose : int Verbosity level. Setting verbose > 0 will display additional information depending on the solver used. random_state : int, RandomState instance or None, optional, default None The seed of the pseudo random number generator to use when shuffling the data. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Used when ``solver`` == 'sag'. return_n_iter : boolean, default False If True, the method also returns `n_iter`, the actual number of iteration performed by the solver. .. versionadded:: 0.17 return_intercept : boolean, default False If True and if X is sparse, the method also returns the intercept, and the solver is automatically changed to 'sag'. This is only a temporary fix for fitting the intercept with sparse data. For dense data, use sklearn.linear_model._preprocess_data before your regression. .. versionadded:: 0.17 Returns ------- coef : array, shape = [n_features] or [n_targets, n_features] Weight vector(s). n_iter : int, optional The actual number of iteration performed by the solver. Only returned if `return_n_iter` is True. intercept : float or array, shape = [n_targets] The intercept of the model. Only returned if `return_intercept` is True and if X is a scipy sparse array. Notes ----- This function won't compute the intercept. sagrhzIn Ridge, only 'sag' solver can currently fit the intercept when X is sparse. Solver has been automatically changed into 'sag'.)risagacsrr:) accept_sparserorderFF)r ensure_2drmcsccoo)rlr)rror zTarget y has the wrong shape %srTz:Number of samples in X and y does not correspond: %d != %dNcholesky sparse_cgz)Sample weights must be 1D array or scalar)rzENumber of targets and number of penalties do not correspond: %d != %d)rtrsr_r@rirjzSolver %s not understoodr@)rCr_)squaredr;rurrj)Zis_sagaz3SVD solver does not support sparse inputs currently)(rissparser.r/r)float64float32rrrr'ndimr-strreshaperSr asarrayrKrbrepeatr=rBr rFr]rrWrQrmaxr*r>ra enumeraterJintr r,rg)r0r1r2rYsolverr3r$r4 random_state return_n_iterreturn_intercept_dtyper5r6rKZ n_samples_rLrZrAr;rXr\Zmax_squared_sumZ interceptr8Zalpha_irOZinitcoef_n_iter__r r r!ridge_regressionsz                              rc @s$eZdZed ddZd d d ZdS) _BaseRidge?TFNMbP?rhc Cs4||_||_||_||_||_||_||_||_dS)N)r2 fit_intercept normalizecopy_Xr3r$rr) selfr2rrrr3r$rrr r r!__init__sz_BaseRidge.__init__c Cs|jdkrtj}n tjtjg}t||dddg|ddd\}}|dk r\t|jdkr\td|j|||j |j |j |d \}}}}}t |r|j rt|||j||j|j|j|jddd \|_|_|_|j|7_n`. Parameters ---------- alpha : {float, array-like}, shape (n_targets) Regularization strength; must be a positive float. Regularization improves the conditioning of the problem and reduces the variance of the estimates. Larger values specify stronger regularization. Alpha corresponds to ``C^-1`` in other linear models such as LogisticRegression or LinearSVC. If an array is passed, penalties are assumed to be specific to the targets. Hence they must correspond in number. fit_intercept : boolean Whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). normalize : boolean, optional, default False This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. copy_X : boolean, optional, default True If True, X will be copied; else, it may be overwritten. max_iter : int, optional Maximum number of iterations for conjugate gradient solver. For 'sparse_cg' and 'lsqr' solvers, the default value is determined by scipy.sparse.linalg. For 'sag' solver, the default value is 1000. tol : float Precision of the solution. solver : {'auto', 'svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag', 'saga'} Solver to use in the computational routines: - 'auto' chooses the solver automatically based on the type of data. - 'svd' uses a Singular Value Decomposition of X to compute the Ridge coefficients. More stable for singular matrices than 'cholesky'. - 'cholesky' uses the standard scipy.linalg.solve function to obtain a closed-form solution. - 'sparse_cg' uses the conjugate gradient solver as found in scipy.sparse.linalg.cg. As an iterative algorithm, this solver is more appropriate than 'cholesky' for large-scale data (possibility to set `tol` and `max_iter`). - 'lsqr' uses the dedicated regularized least-squares routine scipy.sparse.linalg.lsqr. It is the fastest and uses an iterative procedure. - 'sag' uses a Stochastic Average Gradient descent, and 'saga' uses its improved, unbiased version named SAGA. Both methods also use an iterative procedure, and are often faster than other solvers when both n_samples and n_features are large. Note that 'sag' and 'saga' fast convergence is only guaranteed on features with approximately the same scale. You can preprocess the data with a scaler from sklearn.preprocessing. All last five solvers support both dense and sparse data. However, only 'sag' and 'saga' supports sparse input when `fit_intercept` is True. .. versionadded:: 0.17 Stochastic Average Gradient descent solver. .. versionadded:: 0.19 SAGA solver. random_state : int, RandomState instance or None, optional, default None The seed of the pseudo random number generator to use when shuffling the data. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Used when ``solver`` == 'sag'. .. versionadded:: 0.17 *random_state* to support Stochastic Average Gradient. Attributes ---------- coef_ : array, shape (n_features,) or (n_targets, n_features) Weight vector(s). intercept_ : float | array, shape = (n_targets,) Independent term in decision function. Set to 0.0 if ``fit_intercept = False``. n_iter_ : array or None, shape (n_targets,) Actual number of iterations for each target. Available only for sag and lsqr solvers. Other solvers will return None. .. versionadded:: 0.17 See also -------- RidgeClassifier : Ridge classifier RidgeCV : Ridge regression with built-in cross validation :class:`sklearn.kernel_ridge.KernelRidge` : Kernel ridge regression combines ridge regression with the kernel trick Examples -------- >>> from sklearn.linear_model import Ridge >>> import numpy as np >>> n_samples, n_features = 10, 5 >>> np.random.seed(0) >>> y = np.random.randn(n_samples) >>> X = np.random.randn(n_samples, n_features) >>> clf = Ridge(alpha=1.0) >>> clf.fit(X, y) # doctest: +NORMALIZE_WHITESPACE Ridge(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=None, normalize=False, random_state=None, solver='auto', tol=0.001) ?TFNMbP?rhc s$tt|j||||||||ddS)N)r2rrrr3r$rr)superrr) rr2rrrr3r$rr) __class__r r!rszRidge.__init__cstt|j|||dS)aFit Ridge regression model Parameters ---------- X : {array-like, sparse matrix}, shape = [n_samples, n_features] Training data y : array-like, shape = [n_samples] or [n_samples, n_targets] Target values sample_weight : float or numpy array of shape [n_samples] Individual weights for each sample Returns ------- self : returns an instance of self. )rY)rrr)rr0r1rY)rr r!rsz Ridge.fit)rTFTNrrhN)N)rrr__doc__rr __classcell__r r )rr!rs rc s<eZdZdZdfdd Zdfd d Zed d ZZS)RidgeClassifieraeClassifier using Ridge regression. Read more in the :ref:`User Guide `. Parameters ---------- alpha : float Regularization strength; must be a positive float. Regularization improves the conditioning of the problem and reduces the variance of the estimates. Larger values specify stronger regularization. Alpha corresponds to ``C^-1`` in other linear models such as LogisticRegression or LinearSVC. fit_intercept : boolean Whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). normalize : boolean, optional, default False This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. copy_X : boolean, optional, default True If True, X will be copied; else, it may be overwritten. max_iter : int, optional Maximum number of iterations for conjugate gradient solver. The default value is determined by scipy.sparse.linalg. tol : float Precision of the solution. class_weight : dict or 'balanced', optional Weights associated with classes in the form ``{class_label: weight}``. If not given, all classes are supposed to have weight one. The "balanced" mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as ``n_samples / (n_classes * np.bincount(y))`` solver : {'auto', 'svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag', 'saga'} Solver to use in the computational routines: - 'auto' chooses the solver automatically based on the type of data. - 'svd' uses a Singular Value Decomposition of X to compute the Ridge coefficients. More stable for singular matrices than 'cholesky'. - 'cholesky' uses the standard scipy.linalg.solve function to obtain a closed-form solution. - 'sparse_cg' uses the conjugate gradient solver as found in scipy.sparse.linalg.cg. As an iterative algorithm, this solver is more appropriate than 'cholesky' for large-scale data (possibility to set `tol` and `max_iter`). - 'lsqr' uses the dedicated regularized least-squares routine scipy.sparse.linalg.lsqr. It is the fastest and uses an iterative procedure. - 'sag' uses a Stochastic Average Gradient descent, and 'saga' uses its unbiased and more flexible version named SAGA. Both methods use an iterative procedure, and are often faster than other solvers when both n_samples and n_features are large. Note that 'sag' and 'saga' fast convergence is only guaranteed on features with approximately the same scale. You can preprocess the data with a scaler from sklearn.preprocessing. .. versionadded:: 0.17 Stochastic Average Gradient descent solver. .. versionadded:: 0.19 SAGA solver. random_state : int, RandomState instance or None, optional, default None The seed of the pseudo random number generator to use when shuffling the data. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Used when ``solver`` == 'sag'. Attributes ---------- coef_ : array, shape (n_features,) or (n_classes, n_features) Weight vector(s). intercept_ : float | array, shape = (n_targets,) Independent term in decision function. Set to 0.0 if ``fit_intercept = False``. n_iter_ : array or None, shape (n_targets,) Actual number of iterations for each target. Available only for sag and lsqr solvers. Other solvers will return None. Examples -------- >>> from sklearn.datasets import load_breast_cancer >>> from sklearn.linear_model import RidgeClassifier >>> X, y = load_breast_cancer(return_X_y=True) >>> clf = RidgeClassifier().fit(X, y) >>> clf.score(X, y) # doctest: +ELLIPSIS 0.9595... See also -------- Ridge : Ridge regression RidgeClassifierCV : Ridge classifier with built-in cross validation Notes ----- For multi-class classification, n_class classifiers are trained in a one-versus-all approach. Concretely, this is implemented by taking advantage of the multi-variate response support in Ridge. ?TFNMbP?rhc s*tt|j|||||||| d||_dS)N)r2rrrr3r$rr)rrr class_weight) rr2rrrr3r$rrr)rr r!r#s  zRidgeClassifier.__init__cst||dddgddtddd|_|j|}|jjd sLt|dd }ntd |jj |j r~|d krnd }|t |j |}t t |j|||d|S)aFit Ridge regression model. Parameters ---------- X : {array-like, sparse matrix}, shape = [n_samples,n_features] Training data y : array-like, shape = [n_samples] Target values sample_weight : float or numpy array of shape (n_samples,) Sample weight. .. versionadded:: 0.17 *sample_weight* support to Classifier. Returns ------- self : returns an instance of self. rkrprqT)rlrrrr) pos_label neg_label multilabel)r/z-%s doesn't support multi-label classificationNg?)rY)rr_label_binarizer fit_transformy_type_ startswithrr-rrrrrrr)rr0r1rYY)rr r!r,s   zRidgeClassifier.fitcCs|jjS)N)rclasses_)rr r r!rXszRidgeClassifier.classes_) rTFTNrNrhN)N) rrrrrrpropertyrrr r )rr!rs v,rc@sxeZdZdZdddZddd Zd d Zd d ZddZddZ ddZ d ddZ ddZ ddZ ddZd!ddZdS)" _RidgeGCVa,Ridge regression with built-in Generalized Cross-Validation It allows efficient Leave-One-Out cross-validation. This class is not intended to be used directly. Use RidgeCV instead. Notes ----- We want to solve (K + alpha*Id)c = y, where K = X X^T is the kernel matrix. Let G = (K + alpha*Id)^-1. Dual solution: c = Gy Primal solution: w = X^T c Compute eigendecomposition K = Q V Q^T. Then G = Q (V + alpha*Id)^-1 Q^T, where (V + alpha*Id) is diagonal. It is thus inexpensive to inverse for many alphas. Let loov be the vector of prediction values for each example when the model was fitted with all examples but this example. loov = (KGY - diag(KG)Y) / diag(I-KG) Let looe be the vector of prediction errors for each example when the model was fitted with all examples but this example. looe = y - loov = c / diag(G) References ---------- http://cbcl.mit.edu/publications/ps/MIT-CSAIL-TR-2007-025.pdf https://www.mit.edu/~9.520/spring07/Classes/rlsslides.pdf g?g?g$@TFNcCs4t||_||_||_||_||_||_||_dS)N) r)r|alphasrrscoringrgcv_modestore_cv_values)rrrrrrrrr r r!rs z_RidgeGCV.__init__cCsHt||jdd}|r"|t|7}t|\}}t|j|}|||fS)NT)rC)r rFr)Z ones_likerZeighr`)rr0r1centered_kernelrXvQQT_yr r r! _pre_computes z_RidgeGCV._pre_computecCs||djddS)Nr rr)axis)sum)rZv_primerr r r! _decomp_diagsz_RidgeGCV._decomp_diagcCs:t|jdkr2|tdftjft|jd}||S)Nr)rGr'slicer)rV)rDBr r r! _diag_dots$z_RidgeGCV._diag_dotc Csld||}t|ddk}d||<t||||}|||} t|jdkrd| ddtjf} | |fS)zHelper function to avoid code duplication between self._errors and self._values. Notes ----- We don't construct matrix G, instead compute action on y & diagonal. g?rg-q=rN)r)varr`rrrGr'rV) rr2r1rrrwconstant_columncG_diagr r r!_errors_and_values_helpers  z#_RidgeGCV._errors_and_values_helpercCs&||||||\}}||d|fS)Nr )r)rr2r1rrrrrr r r!_errorssz_RidgeGCV._errorscCs&||||||\}}||||fS)N)r)rr2r1rrrrrr r r!_valuessz_RidgeGCV._valuesc Csht|rtd|r4t|t|jddff}tj|dd\}}}|d}t |j |}|||fS)Nz%SVD not supported for sparse matricesrr)r^r ) rrvr,r)ZhstackZonesr'rr_r`rF) rr0r1rrcrdrrUT_yr r r!_pre_compute_svds z_RidgeGCV._pre_compute_svdc Cst|ddk}||d|d}|d ||<t|||||d|}||||d} t|jdkr| ddtjf} | |fS)ziHelper function to avoid code duplication between self._errors_svd and self._values_svd. rg-q=rrrN)r)rr`rrrGr'rV) rr2r1rrcrrrrrr r r!_errors_and_values_svd_helpers z'_RidgeGCV._errors_and_values_svd_helpercCs&||||||\}}||d|fS)Nr )r)rr2r1rrcrrrr r r! _errors_svdsz_RidgeGCV._errors_svdcCs&||||||\}}||||fS)N)r)rr2r1rrcrrrr r r! _values_svdsz_RidgeGCV._values_svdcst|dddgtjddd\}|dk r>t|ts>t|dd}|j\}}tj||j |j |j |d \}}}}|j } t t|} | dks| d krt|s||ks| rd } qd } n| d kr| rtd d } | d kr|j} |j} |j} n*| d kr|j} |j} |j} n td| |dk r.t||\}t| o@|j }| ||\}}}t jdkrhdnjd}t||t |jfg}t||jdddk}t|jdkrtd|jxnt |jD]`\}}|r| t||||\}}n| t||||\}}|!dd|f<|"|qW|rPj#dd$}nHdddd_%dd_&fddt't |jD}t(|}|j||_)|||_*t+|j*j,||_-|.||||j/rt jdkr|t |jf}n||t |jf}0||_1|S)aFit Ridge regression model Parameters ---------- X : {array-like, sparse matrix}, shape = [n_samples, n_features] Training data y : array-like, shape = [n_samples] or [n_samples, n_targets] Target values. Will be cast to X's dtype if necessary sample_weight : float or array-like of shape [n_samples] Sample weight Returns ------- self : object rkrprqT)rrrNF)ro)rYrhZeigenr_zFnon-uniform sample weights unsupported for svd, forcing usage of eigenzbad gcv_mode "%s"r)rZ allow_nonerzIalphas cannot be negative. Got {} containing some negative value instead.)rcSsdS)Nr r r r r!identity_estimator=sz)_RidgeGCV.fit..identity_estimatorcSs|S)Nr ) y_predictr r r!?sz_RidgeGCV.fit..cSs|S)Nr )rr r r!r@scs(g|] }dd|fqS)N)rK).0r8) cv_valuesrscorerr1r r! Bsz!_RidgeGCV.fit..)2rr)rwrUfloatrr'rrrrrrrGrrvr.r/rrrrrrr-r rarrranyformatrrKappendZmeanZargminZdecision_functionZpredictr+Zargmaxalpha_Z dual_coef_r rFrrrr{ cv_values_)rr0r1rYr5r6rrrrZwith_swrrrrrrrZn_yr:errorr8r2outrZbestZcv_values_shaper )rrrr1r!rs|              z _RidgeGCV.fit)rTFNTNF)T)T)N)rrrrrrrrrrrrrrrrr r r r!r]s %  rc@s eZdZd ddZd ddZdS) _BaseRidgeCVg?g?g$@TFNcCs4t||_||_||_||_||_||_||_dS)N) r)r|rrrrcvrr)rrrrrrrrr r r!rWs z_BaseRidgeCV.__init__cCs|jdkrRt|j|j|j|j|j|jd}|j|||d|j |_ |jr|j |_ nX|jr`t dd|ji}t t |j|jd||j|jd}|j|||d|j}|jj|_ |j|_|j|_|S)aFit Ridge regression model Parameters ---------- X : array-like, shape = [n_samples, n_features] Training data y : array-like, shape = [n_samples] or [n_samples, n_targets] Target values. Will be cast to X's dtype if necessary sample_weight : float or array-like of shape [n_samples] Sample weight Returns ------- self : object N)rrrrr)rYz3cv!=None and store_cv_values=True are incompatibler2)rr)rr)rrrrrrrrrrrr-rrZbest_estimator_r2rr)rr0r1rYZ estimatorZ parametersZgsr r r!rcs.     z_BaseRidgeCV.fit)rTFNNNF)N)rrrrrr r r r!rVs  rc@seZdZdZdS)RidgeCVaRidge regression with built-in cross-validation. See glossary entry for :term:`cross-validation estimator`. By default, it performs Generalized Cross-Validation, which is a form of efficient Leave-One-Out cross-validation. Read more in the :ref:`User Guide `. Parameters ---------- alphas : numpy array of shape [n_alphas] Array of alpha values to try. Regularization strength; must be a positive float. Regularization improves the conditioning of the problem and reduces the variance of the estimates. Larger values specify stronger regularization. Alpha corresponds to ``C^-1`` in other linear models such as LogisticRegression or LinearSVC. fit_intercept : boolean Whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). normalize : boolean, optional, default False This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. scoring : string, callable or None, optional, default: None A string (see model evaluation documentation) or a scorer callable object / function with signature ``scorer(estimator, X, y)``. cv : int, cross-validation generator or an iterable, optional Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the efficient Leave-One-Out cross-validation - integer, to specify the number of folds. - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. For integer/None inputs, if ``y`` is binary or multiclass, :class:`sklearn.model_selection.StratifiedKFold` is used, else, :class:`sklearn.model_selection.KFold` is used. Refer :ref:`User Guide ` for the various cross-validation strategies that can be used here. gcv_mode : {None, 'auto', 'svd', eigen'}, optional Flag indicating which strategy to use when performing Generalized Cross-Validation. Options are:: 'auto' : use svd if n_samples > n_features or when X is a sparse matrix, otherwise use eigen 'svd' : force computation via singular value decomposition of X (does not work for sparse matrices) 'eigen' : force computation via eigendecomposition of X^T X The 'auto' mode is the default and is intended to pick the cheaper option of the two depending upon the shape and format of the training data. store_cv_values : boolean, default=False Flag indicating if the cross-validation values corresponding to each alpha should be stored in the ``cv_values_`` attribute (see below). This flag is only compatible with ``cv=None`` (i.e. using Generalized Cross-Validation). Attributes ---------- cv_values_ : array, shape = [n_samples, n_alphas] or shape = [n_samples, n_targets, n_alphas], optional Cross-validation values for each alpha (if ``store_cv_values=True`` and ``cv=None``). After ``fit()`` has been called, this attribute will contain the mean squared errors (by default) or the values of the ``{loss,score}_func`` function (if provided in the constructor). coef_ : array, shape = [n_features] or [n_targets, n_features] Weight vector(s). intercept_ : float | array, shape = (n_targets,) Independent term in decision function. Set to 0.0 if ``fit_intercept = False``. alpha_ : float Estimated regularization parameter. Examples -------- >>> from sklearn.datasets import load_diabetes >>> from sklearn.linear_model import RidgeCV >>> X, y = load_diabetes(return_X_y=True) >>> clf = RidgeCV(alphas=[1e-3, 1e-2, 1e-1, 1]).fit(X, y) >>> clf.score(X, y) # doctest: +ELLIPSIS 0.5166... See also -------- Ridge : Ridge regression RidgeClassifier : Ridge classifier RidgeClassifierCV : Ridge classifier with built-in cross validation N)rrrrr r r r!rskrcs8eZdZdZd fdd Zd dd Zed d ZZS)RidgeClassifierCVaRidge classifier with built-in cross-validation. See glossary entry for :term:`cross-validation estimator`. By default, it performs Generalized Cross-Validation, which is a form of efficient Leave-One-Out cross-validation. Currently, only the n_features > n_samples case is handled efficiently. Read more in the :ref:`User Guide `. Parameters ---------- alphas : numpy array of shape [n_alphas] Array of alpha values to try. Regularization strength; must be a positive float. Regularization improves the conditioning of the problem and reduces the variance of the estimates. Larger values specify stronger regularization. Alpha corresponds to ``C^-1`` in other linear models such as LogisticRegression or LinearSVC. fit_intercept : boolean Whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). normalize : boolean, optional, default False This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. scoring : string, callable or None, optional, default: None A string (see model evaluation documentation) or a scorer callable object / function with signature ``scorer(estimator, X, y)``. cv : int, cross-validation generator or an iterable, optional Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the efficient Leave-One-Out cross-validation - integer, to specify the number of folds. - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. Refer :ref:`User Guide ` for the various cross-validation strategies that can be used here. class_weight : dict or 'balanced', optional Weights associated with classes in the form ``{class_label: weight}``. If not given, all classes are supposed to have weight one. The "balanced" mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as ``n_samples / (n_classes * np.bincount(y))`` store_cv_values : boolean, default=False Flag indicating if the cross-validation values corresponding to each alpha should be stored in the ``cv_values_`` attribute (see below). This flag is only compatible with ``cv=None`` (i.e. using Generalized Cross-Validation). Attributes ---------- cv_values_ : array, shape = [n_samples, n_targets, n_alphas], optional Cross-validation values for each alpha (if ``store_cv_values=True`` and ``cv=None``). After ``fit()`` has been called, this attribute will contain the mean squared errors (by default) or the values of the ``{loss,score}_func`` function (if provided in the constructor). coef_ : array, shape = [n_features] or [n_targets, n_features] Weight vector(s). intercept_ : float | array, shape = (n_targets,) Independent term in decision function. Set to 0.0 if ``fit_intercept = False``. alpha_ : float Estimated regularization parameter Examples -------- >>> from sklearn.datasets import load_breast_cancer >>> from sklearn.linear_model import RidgeClassifierCV >>> X, y = load_breast_cancer(return_X_y=True) >>> clf = RidgeClassifierCV(alphas=[1e-3, 1e-2, 1e-1, 1]).fit(X, y) >>> clf.score(X, y) # doctest: +ELLIPSIS 0.9630... See also -------- Ridge : Ridge regression RidgeClassifier : Ridge classifier RidgeCV : Ridge regression with built-in cross validation Notes ----- For multi-class classification, n_class classifiers are trained in a one-versus-all approach. Concretely, this is implemented by taking advantage of the multi-variate response support in Ridge. g?g?g$@TFNcs&tt|j||||||d||_dS)N)rrrrrr)rrrr)rrrrrrrr)rr r!rjs  zRidgeClassifierCV.__init__cCst||dddgddtddd|_|j|}|jjd sJt|dd }|jrl|d kr\d }|t|j|}t j ||||d |S)aFit the ridge classifier. Parameters ---------- X : array-like, shape (n_samples, n_features) Training vectors, where n_samples is the number of samples and n_features is the number of features. y : array-like, shape (n_samples,) Target values. Will be cast to X's dtype if necessary sample_weight : float or numpy array of shape (n_samples,) Sample weight. Returns ------- self : object rkrprqT)rlrrrr)rrr)r/Ng?)rY) rrrrrrrrrrr)rr0r1rYrr r r!rrs  zRidgeClassifierCV.fitcCs|jjS)N)rr)rr r r!rszRidgeClassifierCV.classes_)rTFNNNF)N) rrrrrrrrrr r )rr!rs g %r)Nrr)Nr)NF)NrhNrrNFF)4rabcrrr.Znumpyr)ZscipyrrZ scipy.sparser(baserrr rir r Z utils.extmathr rZutilsrrrrrZ preprocessingrZmodel_selectionrZ externalsrZmetrics.scorerr exceptionsrr=rBrQr]rgrZwith_metaclassrrrrrrrr r r r!sR                   9  =  5$3z<o