B \L@sdZddlmZddlmZmZddlZddlZddlZ ddl m Z ddl mZddl m Z ddlmZdd lmZmZdd lmZmZmZdd lmZmZdd lmZdd lmZddlmZddl m!Z!m"Z"ddl#m$Z$ddl%m&Z&m'Z'ddlm(Z(ddl)m*Z*ddl+m,Z-dZ.d(ddZ/d)ddZ0ddZ1Gddde2eeZ3Gd d!d!eZ4Gd"d#d#e5Z6Gd$d%d%e3eZ7d*d&d'Z8dS)+z Generalized Linear models. )division)ABCMetaabstractmethodN)linalg)sparse)six)Paralleldelayed) BaseEstimatorClassifierMixinRegressorMixin) check_array check_X_y) FLOAT_DTYPES)check_random_state)safe_sparse_dot)mean_variance_axisinplace_column_scale) sparse_lsqr) ArrayDataset CSRDataset)check_is_fitted)NotFittedError) normalizeg{Gz?cCsdt|}|dttjj}t|rHt|j |j |j |||d}t }nt ||||d}d}||fS)aCreate ``Dataset`` abstraction for sparse and dense inputs. This also returns the ``intercept_decay`` which is different for sparse datasets. Parameters ---------- X : array_like, shape (n_samples, n_features) Training data y : array_like, shape (n_samples, ) Target values. sample_weight : numpy array of shape (n_samples,) The weight of each sample random_state : int, RandomState instance or None (default) Determines random number generation for dataset shuffling and noise. Pass an int for reproducible output across multiple function calls. See :term:`Glossary `. Returns ------- dataset The ``Dataset`` abstraction intercept_decay The intercept decay )seedg?)rZrandintnpZiinfoZint32maxspissparserdataZindptrindicesSPARSE_INTERCEPT_DECAYr)Xy sample_weightZ random_staterngrZdatasetZintercept_decayr(8lib/python3.7/site-packages/sklearn/linear_model/base.py make_dataset0s r*FTc Cst|tjrd}|r*t||ddgtd}n$|rNt|rB|}n |jdd}tj ||j d}|rNt|rt |dd \}} |s|j d|dd<|r| |j d9} t| | } ~ d | | dk<t|d | ntj|j d |j d} nJtj|d|d }||8}|rt|dd dd\}} ntj|j d |j d} tj|d|d } || }n\tj|j d |j d}tj|j d |j d} |jd kr|j d} ntj|j d |j d} |||| | fS)aS Centers data to have mean zero along axis 0. If fit_intercept=False or if the X is a sparse matrix, no centering is done, but normalization can still be applied. The function returns the statistics necessary to reconstruct the input data, which are X_offset, y_offset, X_scale, such that the output X = (X - X_offset) / X_scale X_scale is the L2 norm of X - X_offset. If sample_weight is not None, then the weighted mean of X and y is zero, and not the mean itself. If return_mean=True, the mean, eventually weighted, is returned, independently of whether X was centered (option used for optimization with sparse data in coordinate_descend). This is here because nearly all linear models will want their data to be centered. This function also systematically makes y consistent with X.dtype Ncsrcsc)copy accept_sparsedtypeK)order)r/r)axisrg?)r2ZweightsFT)r2r-Z return_norm) isinstancenumbersNumberrrrr r-rZasarrayr/rtypeshapesqrtronesZaverage f_normalizezerosndim) r$r% fit_interceptrr-r& return_mean check_inputX_offsetZX_varX_scaley_offsetr(r(r)_preprocess_data]sH           rCcCs^|jd}tj||t|jd}t|}tj|df||fd}t||}t||}||fS)z+Rescale data so as to support sample_weightr)r/)r7) r7rZfullZarrayr/r8rZ dia_matrixr)r$r%r& n_samplesZ sw_matrixr(r(r) _rescale_datas      rEc@s<eZdZdZeddZddZddZee Z dd Z d S) LinearModelzBase class for Linear ModelscCsdS)z Fit model.Nr()selfr$r%r(r(r)fitszLinearModel.fitcCs4t|dt|dddgd}t||jjdd|jS)Ncoef_r+r,coo)r.T) dense_output)rrrrIT intercept_)rGr$r(r(r)_decision_functions  zLinearModel._decision_functioncCs ||S)aPredict using the linear model Parameters ---------- X : array_like or sparse matrix, shape (n_samples, n_features) Samples. Returns ------- C : array, shape (n_samples,) Returns predicted values. )rN)rGr$r(r(r)predicts zLinearModel.predictcCs4|jr*|j||_|t||jj|_nd|_dS)zSet the intercept_ gN)r=rIrdotrLrM)rGr@rBrAr(r(r)_set_intercepts zLinearModel._set_interceptN) __name__ __module__ __qualname____doc__rrHrNrO staticmethodrCrQr(r(r(r)rFs  rFc@s(eZdZdZddZddZddZdS) LinearClassifierMixinzRMixin for linear classifiers. Handles prediction for sparse and dense X. cCst|dr|jdkr*tddt|jit|dd}|jjd}|jd|krftd|jd|ft||jj d d |j }|jddkr| S|S) aNPredict confidence scores for samples. The confidence score for a sample is the signed distance of that sample to the hyperplane. Parameters ---------- X : array_like or sparse matrix, shape (n_samples, n_features) Samples. Returns ------- array, shape=(n_samples,) if n_classes == 2 else (n_samples, n_classes) Confidence scores per (sample, class) combination. In the binary case, confidence score for self.classes_[1] where >0 means this class would be predicted. rINz(This %(name)s instance is not fitted yetnamer+)r.rz*X has %d features per sample; expecting %dT)rK) hasattrrIrr6rRrr7 ValueErrorrrLrMravel)rGr$ n_featuresscoresr(r(r)decision_functions   z'LinearClassifierMixin.decision_functioncCs@||}t|jdkr*|dktj}n |jdd}|j|S)a&Predict class labels for samples in X. Parameters ---------- X : array_like or sparse matrix, shape (n_samples, n_features) Samples. Returns ------- C : array, shape [n_samples] Predicted class label per sample. rr)r2)r^lenr7ZastyperintZargmaxZclasses_)rGr$r]r"r(r(r)rO s  zLinearClassifierMixin.predictcCsx||}|d9}t|||d7}t|||jdkrPtd||gjS||jdd|j ddf}|SdS)zProbability estimation for OvR logistic regression. Positive class probabilities are computed as 1. / (1. + np.exp(-self.decision_function(X))); multiclass is handled by normalizing that over all classes. r)r2rN) r^rZexpZ reciprocalr<vstackrLsumZreshaper7)rGr$Zprobr(r(r)_predict_proba_lr s     z'LinearClassifierMixin._predict_proba_lrN)rRrSrTrUr^rOrdr(r(r(r)rWs!rWc@s eZdZdZddZddZdS)SparseCoefMixinzlMixin for converting coef_ to and from CSR format. L1-regularizing estimators should inherit this. cCs.d}t|d|dt|jr*|j|_|S)aConvert coefficient matrix to dense array format. Converts the ``coef_`` member (back) to a numpy.ndarray. This is the default format of ``coef_`` and is required for fitting, so calling this method is only required on models that have previously been sparsified; otherwise, it is a no-op. Returns ------- self : estimator z6Estimator, %(name)s, must be fitted before densifying.rI)msg)rrr rIZtoarray)rGrfr(r(r)densify:s   zSparseCoefMixin.densifycCs$d}t|d|dt|j|_|S)avConvert coefficient matrix to sparse format. Converts the ``coef_`` member to a scipy.sparse matrix, which for L1-regularized models can be much more memory- and storage-efficient than the usual numpy.ndarray representation. The ``intercept_`` member is not converted. Notes ----- For non-sparse models, i.e. when there are not many zeros in ``coef_``, this may actually *increase* memory usage, so use this method with care. A rule of thumb is that the number of zero elements, which can be computed with ``(coef_ == 0).sum()``, must be more than 50% for this to provide significant benefits. After calling this method, further fitting with the partial_fit method (if any) will not work until you call densify. Returns ------- self : estimator z7Estimator, %(name)s, must be fitted before sparsifying.rI)rf)rrZ csr_matrixrI)rGrfr(r(r)sparsifyLszSparseCoefMixin.sparsifyN)rRrSrTrUrgrhr(r(r(r)re4srec@s$eZdZdZd ddZd ddZdS) LinearRegressiona Ordinary least squares Linear Regression. Parameters ---------- fit_intercept : boolean, optional, default True whether to calculate the intercept for this model. If set to False, no intercept will be used in calculations (e.g. data is expected to be already centered). normalize : boolean, optional, default False This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. copy_X : boolean, optional, default True If True, X will be copied; else, it may be overwritten. n_jobs : int or None, optional (default=None) The number of jobs to use for the computation. This will only provide speedup for n_targets > 1 and sufficient large problems. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Attributes ---------- coef_ : array, shape (n_features, ) or (n_targets, n_features) Estimated coefficients for the linear regression problem. If multiple targets are passed during the fit (y 2D), this is a 2D array of shape (n_targets, n_features), while if only one target is passed, this is a 1D array of length n_features. intercept_ : array Independent term in the linear model. Examples -------- >>> import numpy as np >>> from sklearn.linear_model import LinearRegression >>> X = np.array([[1, 1], [1, 2], [2, 2], [2, 3]]) >>> # y = 1 * x_0 + 2 * x_1 + 3 >>> y = np.dot(X, np.array([1, 2])) + 3 >>> reg = LinearRegression().fit(X, y) >>> reg.score(X, y) 1.0 >>> reg.coef_ array([1., 2.]) >>> reg.intercept_ # doctest: +ELLIPSIS 3.0000... >>> reg.predict(np.array([[3, 5]])) array([16.]) Notes ----- From the implementation point of view, this is just plain Ordinary Least Squares (scipy.linalg.lstsq) wrapped as a predictor object. TFNcCs||_||_||_||_dS)N)r=rcopy_Xn_jobs)rGr=rrjrkr(r(r)__init__szLinearRegression.__init__c s^|j}tdddgddd\|dk rBt|jdkrBtd|j|j|j|j |d \}}}|dk rt |\t r jd krt }|d |_|d |_nTt|d fddtjdD} tdd| D|_tdd| D|_n&t\|_|_|_|_|jj|_jdkrLt|j|_|||||S)aC Fit linear model. Parameters ---------- X : array-like or sparse matrix, shape (n_samples, n_features) Training data y : array_like, shape (n_samples, n_targets) Target values. Will be cast to X's dtype if necessary sample_weight : numpy array of shape [n_samples] Individual weights for each sample .. versionadded:: 0.17 parameter *sample_weight* support to LinearRegression. Returns ------- self : returns an instance of self. r+r,rJT)r.Z y_numericZ multi_outputNrz)Sample weights must be 1D array or scalar)r=rr-r&rr)rkc3s,|]$}ttdd|fVqdS)N)r rr[).0j)r$r%r(r) sz'LinearRegression.fit..cSsg|] }|dqS)rr()rnoutr(r(r) sz(LinearRegression.fit..cSsg|] }|dqS)rmr()rnrqr(r(r)rrs)rkrrZ atleast_1dr<rZrCr=rrjrErr rrIZ _residuesr ranger7rbrZlstsqZrank_Z singular_rLr[rQ) rGr$r%r&Zn_jobs_r@rBrArqZoutsr()r$r%r)rHs4         zLinearRegression.fit)TFTN)N)rRrSrTrUrlrHr(r(r(r)rijs> ric Cs|j\}} t|r:d}t||||dd|d\}}} } } nt||||||d\}}} } } t|dr|rxt| t| r|rt| t| st dt d}d}t |t jr|dkr|| k}|dkrtj| | f|jd d }tj|j||d t|dsd}t|dr|dkrt|j|jgg} |jd krVtj| | d d }tj|j||d n2|jd }tj| |f| d d }tj|j||jd ||| | | ||fS)z6Aux function used at beginning of fit in linear modelsFT)r=rr-r>r?)r=rr-r?Z __array__zlGram matrix was provided but X was centered to fit intercept, or X was normalized : recomputing Gram matrix.autoNC)r7r/r1)rqrF)r7rZ isspmatrixrCrYrZallcloser;r9warningswarn UserWarningr3rZ string_typesemptyr/rPrLZfind_common_typer<)r$r%ZXyZ precomputerr=r-r?rDr\r@rBrAZ common_dtypeZ n_targetsr(r(r)_pre_fitsD        r{)N)FTNFT)T)9rUZ __future__rabcrrr4rwZnumpyrZ scipy.sparserrZscipyrZ externalsrZ utils._joblibr r baser r r ZutilsrrZutils.validationrrZ utils.extmathrZutils.sparsefuncsrrZ utils.fixesrZutils.seq_datasetrrr exceptionsrZpreprocessing.datarr:r#r*rCrEZwith_metaclassrFrWobjectrerir{r(r(r(r)s@             - O +O6