B \@sdZddlZddlmZddlZddlmZddlm Z ddl m Z m Z dd l m Z dd lmZmZmZdd lmZdd lmZmZd ZdddZd ddZd!ddZd"ddZGddde e Zd#ddZGddde e ZdS)$z'Orthogonal matching pursuit algorithms N)sqrt)linalg)get_lapack_funcs) LinearModel_pre_fit)RegressorMixin)as_float_array check_array check_X_y)check_cv)Paralleldelayedz Orthogonal matching pursuit ended prematurely due to linear dependence in the dictionary. The requested precision might not have been met. TFcCs|r|d}n t|}t|jj}td|f\}}td|f\} t |j |} |} t d} d} t |j d}|dk r|j dn|}tj ||f|jd}|rt|}x*ttt |j | }|| ks| |d|krtjttdd P| dkrt |ddd| fj |dd|f|| d| f<tj|d| d| f|| d| fddd d d ||| d| fd}t|dd|fd|}||krtjttdd Pt||| | f<nt|dd|f|d <||j | |j |\|j | <|j |<| || | | | <| |<|||| || <||<| d7} | |d| d| f| d| d d d\} }|r| |d| | df<|t |ddd| f| } |dk r|| d|krPq| |krPqW|r| |d| |ddd| f| fS| |d| | fSdS)avOrthogonal Matching Pursuit step using the Cholesky decomposition. Parameters ---------- X : array, shape (n_samples, n_features) Input dictionary. Columns are assumed to have unit norm. y : array, shape (n_samples,) Input targets n_nonzero_coefs : int Targeted number of non-zero elements tol : float Targeted squared error, if not None overrides n_nonzero_coefs. copy_X : bool, optional Whether the design matrix X must be copied by the algorithm. A false value is only helpful if X is already Fortran-ordered, otherwise a copy is made anyway. return_path : bool, optional. Default: False Whether to return every value of the nonzero coefficients along the forward path. Useful for cross-validation. Returns ------- gamma : array, shape (n_nonzero_coefs,) Non-zero elements of the solution idx : array, shape (n_nonzero_coefs,) Indices of the positions of the elements in gamma within the solution vector coef : array, shape (n_features, n_nonzero_coefs) The first k values of column k correspond to the coefficient value for the active features at that step. The lower left triangle contains garbage. Only returned if ``return_path=True``. n_active : int Number of active features at convergence. F)nrm2swap)potrsrrN)dtyper) stacklevelTF)translower overwrite_b check_finite)rr)rr)copynpasfortranarrayfinforepsrget_blas_funcsrdotTemptyarangeshape empty_likeargmaxabswarningswarn prematureRuntimeWarningsolve_triangularZnormr)Xyn_nonzero_coefstolcopy_X return_path min_floatrrralphaZresidualgamman_activeindices max_featuresLcoefslamvLkk_r?7lib/python3.7/site-packages/sklearn/linear_model/omp.py _cholesky_ompsd,     6 &   $rAcCs8|r|dnt|}|s$|jjs,|}t|jj}t d|f\} } t d|f\} t t |} |} |}d}t d}d}|dk rt |n|}tj ||f|jd}d|d<|rt|}x4tt| }||ks| |d |krtjttd d P|dkr||d|f||d|f<tj|d|d|f||d|fdd d dd| ||d|fd }|||f|}||krtjttd d Pt||||f<nt|||f|d<| ||||\||<||<| |j||j|\|j|<|j|<| || || |<| |<||||||<||<|d 7}| |d|d|f|d|d dd\}}|r||d||d f<t|ddd|f|}||} |dk r||7}t||d|}||8}t||krPq||krPqW|r"|| d||ddd|f|fS|| d||fSdS)a|Orthogonal Matching Pursuit step on a precomputed Gram matrix. This function uses the Cholesky decomposition method. Parameters ---------- Gram : array, shape (n_features, n_features) Gram matrix of the input data matrix Xy : array, shape (n_features,) Input targets n_nonzero_coefs : int Targeted number of non-zero elements tol_0 : float Squared norm of y, required if tol is not None. tol : float Targeted squared error, if not None overrides n_nonzero_coefs. copy_Gram : bool, optional Whether the gram matrix must be copied by the algorithm. A false value is only helpful if it is already Fortran-ordered, otherwise a copy is made anyway. copy_Xy : bool, optional Whether the covariance vector Xy must be copied by the algorithm. If False, it may be overwritten. return_path : bool, optional. Default: False Whether to return every value of the nonzero coefficients along the forward path. Useful for cross-validation. Returns ------- gamma : array, shape (n_nonzero_coefs,) Non-zero elements of the solution idx : array, shape (n_nonzero_coefs,) Indices of the positions of the elements in gamma within the solution vector coefs : array, shape (n_features, n_nonzero_coefs) The first k values of column k correspond to the coefficient value for the active features at that step. The lower left triangle contains garbage. Only returned if ``return_path=True``. n_active : int Number of active features at convergence. r)rr)rrN)rg?)rrr)rrTF)rrrr)rr)rrrflags writeablerrrrrrr#lenr"r%r&r'r(r)r*r+r,rr!r inner)GramXyr/Ztol_0r0 copy_Gramcopy_Xyr2r3rrrr7r4Ztol_currZdeltar5r6r8r9r:r;r<r=r>Zbetar?r?r@ _gram_ompst5     &   $rKc Cst|d|d}d}|jdkr(|dd}t|}|jddkrBd}|dkrj|dkrjttd|jdd}|dk r|d krtd |dkr|d krtd |dkr||jdkrtd |d kr|jd |jdk}|r:t|j |}t |}t|j |} |dk rtj |dd d} nd} t || ||| |d|dS|rdt |jd|jd|jdf} nt |jd|jdf} g} xt|jdD]} t||dd| f||||d}|r(|\}}}}| dddddt|f} xVt|j D]0\}}|d|d| |d|d| |f<qWn|\}}}|| || f<| |qW|jddkrf| d } |rzt| | fSt| SdS)aI Orthogonal Matching Pursuit (OMP) Solves n_targets Orthogonal Matching Pursuit problems. An instance of the problem has the form: When parametrized by the number of non-zero coefficients using `n_nonzero_coefs`: argmin ||y - X\gamma||^2 subject to ||\gamma||_0 <= n_{nonzero coefs} When parametrized by error using the parameter `tol`: argmin ||\gamma||_0 subject to ||y - X\gamma||^2 <= tol Read more in the :ref:`User Guide `. Parameters ---------- X : array, shape (n_samples, n_features) Input data. Columns are assumed to have unit norm. y : array, shape (n_samples,) or (n_samples, n_targets) Input targets n_nonzero_coefs : int Desired number of non-zero entries in the solution. If None (by default) this value is set to 10% of n_features. tol : float Maximum norm of the residual. If not None, overrides n_nonzero_coefs. precompute : {True, False, 'auto'}, Whether to perform precomputations. Improves performance when n_targets or n_samples is very large. copy_X : bool, optional Whether the design matrix X must be copied by the algorithm. A false value is only helpful if X is already Fortran-ordered, otherwise a copy is made anyway. return_path : bool, optional. Default: False Whether to return every value of the nonzero coefficients along the forward path. Useful for cross-validation. return_n_iter : bool, optional default False Whether or not to return the number of iterations. Returns ------- coef : array, shape (n_features,) or (n_features, n_targets) Coefficients of the OMP solution. If `return_path=True`, this contains the whole coefficient path. In this case its shape is (n_features, n_features) or (n_features, n_targets, n_features) and iterating over the last axis yields coefficients in increasing order of active features. n_iters : array-like or int Number of active features across every target. Returned only if `return_n_iter` is set to True. See also -------- OrthogonalMatchingPursuit orthogonal_mp_gram lars_path decomposition.sparse_encode Notes ----- Orthogonal matching pursuit was introduced in S. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Transactions on Signal Processing, Vol. 41, No. 12. (December 1993), pp. 3397-3415. (http://blanche.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf) This implementation is based on Rubinstein, R., Zibulevsky, M. and Elad, M., Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Technical Report - CS Technion, April 2008. http://www.cs.technion.ac.il/~ronrubin/Publications/KSVD-OMP-v2.pdf r)orderrFrTNg?rzEpsilon cannot be negativez$The number of atoms must be positivez>The number of atoms cannot be more than the number of featuresautor)axis)rIrJr2)r1r2)r ndimZreshaper$maxint ValueErrorrr r!rsumorthogonal_mp_gramzerosrangerArE enumerateappendsqueeze)r-r.r/r0 precomputer1r2 return_n_iterGrH norms_squaredcoefn_iterskoutr>idxr:n_iterr6xr?r?r@ orthogonal_mp s`Q     $  .  rfc  CsTt|d|d}t|}|jdkr4|jddkr4d}|jdkr^|ddtjf}|dk r^|g}|sj|jjsr|}|dkr|dkrt dt |}|dk r|dkrt d|dk r|dkrt d |dkr|dkrt d |dkr|t |krt d |rt t ||jdt |f} nt t ||jdf} g} xt |jdD]} t||dd| f||dk rr|| nd||d |d } |r| \} }}}| dddddt |f} xVt|jD]0\}}|d|d| |d|d| |f<qWn| \}}}|| || f<| |qHW|jddkr2| d} |rFt| | fSt| SdS)a Gram Orthogonal Matching Pursuit (OMP) Solves n_targets Orthogonal Matching Pursuit problems using only the Gram matrix X.T * X and the product X.T * y. Read more in the :ref:`User Guide `. Parameters ---------- Gram : array, shape (n_features, n_features) Gram matrix of the input data: X.T * X Xy : array, shape (n_features,) or (n_features, n_targets) Input targets multiplied by X: X.T * y n_nonzero_coefs : int Desired number of non-zero entries in the solution. If None (by default) this value is set to 10% of n_features. tol : float Maximum norm of the residual. If not None, overrides n_nonzero_coefs. norms_squared : array-like, shape (n_targets,) Squared L2 norms of the lines of y. Required if tol is not None. copy_Gram : bool, optional Whether the gram matrix must be copied by the algorithm. A false value is only helpful if it is already Fortran-ordered, otherwise a copy is made anyway. copy_Xy : bool, optional Whether the covariance vector Xy must be copied by the algorithm. If False, it may be overwritten. return_path : bool, optional. Default: False Whether to return every value of the nonzero coefficients along the forward path. Useful for cross-validation. return_n_iter : bool, optional default False Whether or not to return the number of iterations. Returns ------- coef : array, shape (n_features,) or (n_features, n_targets) Coefficients of the OMP solution. If `return_path=True`, this contains the whole coefficient path. In this case its shape is (n_features, n_features) or (n_features, n_targets, n_features) and iterating over the last axis yields coefficients in increasing order of active features. n_iters : array-like or int Number of active features across every target. Returned only if `return_n_iter` is set to True. See also -------- OrthogonalMatchingPursuit orthogonal_mp lars_path decomposition.sparse_encode Notes ----- Orthogonal matching pursuit was introduced in G. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Transactions on Signal Processing, Vol. 41, No. 12. (December 1993), pp. 3397-3415. (http://blanche.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf) This implementation is based on Rubinstein, R., Zibulevsky, M. and Elad, M., Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Technical Report - CS Technion, April 2008. http://www.cs.technion.ac.il/~ronrubin/Publications/KSVD-OMP-v2.pdf r)rLrrTNg?zSGram OMP needs the precomputed norms in order to evaluate the error sum of squares.rzEpsilon cannot be negativez$The number of atoms must be positivez>The number of atoms cannot be more than the number of featuresF)rIrJr2)r rZasarrayrPr$newaxisrCrDrrRrErSrVrWrKrXr!rYrZ)rGrHr/r0r^rIrJr2r\r_r`rarbr>rcr:rdr6rer?r?r@rUsVN     .  rUc@s"eZdZdZd ddZddZdS) OrthogonalMatchingPursuita Orthogonal Matching Pursuit model (OMP) Read more in the :ref:`User Guide `. Parameters ---------- n_nonzero_coefs : int, optional Desired number of non-zero entries in the solution. If None (by default) this value is set to 10% of n_features. tol : float, optional Maximum norm of the residual. If not None, overrides n_nonzero_coefs. fit_intercept : boolean, optional whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). normalize : boolean, optional, default True This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. precompute : {True, False, 'auto'}, default 'auto' Whether to use a precomputed Gram and Xy matrix to speed up calculations. Improves performance when `n_targets` or `n_samples` is very large. Note that if you already have such matrices, you can pass them directly to the fit method. Attributes ---------- coef_ : array, shape (n_features,) or (n_targets, n_features) parameter vector (w in the formula) intercept_ : float or array, shape (n_targets,) independent term in decision function. n_iter_ : int or array-like Number of active features across every target. Examples -------- >>> from sklearn.linear_model import OrthogonalMatchingPursuit >>> from sklearn.datasets import make_regression >>> X, y = make_regression(noise=4, random_state=0) >>> reg = OrthogonalMatchingPursuit().fit(X, y) >>> reg.score(X, y) # doctest: +ELLIPSIS 0.9991... >>> reg.predict(X[:1,]) array([-78.3854...]) Notes ----- Orthogonal matching pursuit was introduced in G. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Transactions on Signal Processing, Vol. 41, No. 12. (December 1993), pp. 3397-3415. (http://blanche.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf) This implementation is based on Rubinstein, R., Zibulevsky, M. and Elad, M., Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Technical Report - CS Technion, April 2008. http://www.cs.technion.ac.il/~ronrubin/Publications/KSVD-OMP-v2.pdf See also -------- orthogonal_mp orthogonal_mp_gram lars_path Lars LassoLars decomposition.sparse_encode OrthogonalMatchingPursuitCV NTrNcCs"||_||_||_||_||_dS)N)r/r0 fit_intercept normalizer[)selfr/r0rirjr[r?r?r@__init__ks z"OrthogonalMatchingPursuit.__init__c Cst||ddd\}}|jd}t||d|j|j|jdd\}}}}}}}|jdkrd|ddtjf}|j dkr|j dkrt t d|d|_ n|j |_ |dkrt|||j |j dddd\} |_nB|j dk rtj|d d d nd} t|||j |j | dddd \} |_| j|_|||||S) aFit the model using X, y as training data. Parameters ---------- X : array-like, shape (n_samples, n_features) Training data. y : array-like, shape (n_samples,) or (n_samples, n_targets) Target values. Will be cast to X's dtype if necessary Returns ------- self : object returns an instance of self. T)Z multi_output y_numericrN)rg?F)r[r1r\rr)rO)rHr/r0r^rIrJr\)r r$rr[rjrirPrrgr/r0rQrRn_nonzero_coefs_rfn_iter_rTrUr!coef_Z_set_intercept) rkr-r.Z n_featuresZX_offsetZy_offsetZX_scalerGrHrpZnorms_sqr?r?r@fitss.    zOrthogonalMatchingPursuit.fit)NNTTrN)__name__ __module__ __qualname____doc__rlrqr?r?r?r@rhsL rhdc Cs(|r$|}|}|}|}|rx|jdd}||8}||8}|jdd} t|dd}|| 8}t|dd}|| 8}|rttj|ddd} t| } |dd| f| | <t|||ddddd} | jd kr| ddtj f} |r| | | | ddtj f<t | j |j |S) aCompute the residues on left-out data for a full LARS path Parameters ----------- X_train : array, shape (n_samples, n_features) The data to fit the LARS on y_train : array, shape (n_samples) The target variable to fit LARS on X_test : array, shape (n_samples, n_features) The data to compute the residues on y_test : array, shape (n_samples) The target variable to compute the residues on copy : boolean, optional Whether X_train, X_test, y_train and y_test should be copied. If False, they may be overwritten. fit_intercept : boolean whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). normalize : boolean, optional, default True This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. max_iter : integer, optional Maximum numbers of iterations to perform, therefore maximum features to include. 100 by default. Returns ------- residues : array, shape (n_samples, max_features) Residues of the prediction on the test data r)rOF)rrNT)r/r0r[r1r2r) rmeanr rrrTZ flatnonzerorfrPrgr r!) ZX_trainZy_trainZX_testZy_testrrirjmax_iterZX_meanZy_meanZnormsZnonzerosr:r?r?r@_omp_path_residuess4-       "ryc@s"eZdZdZd ddZdd ZdS) OrthogonalMatchingPursuitCVa Cross-validated Orthogonal Matching Pursuit model (OMP). See glossary entry for :term:`cross-validation estimator`. Read more in the :ref:`User Guide `. Parameters ---------- copy : bool, optional Whether the design matrix X must be copied by the algorithm. A false value is only helpful if X is already Fortran-ordered, otherwise a copy is made anyway. fit_intercept : boolean, optional whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). normalize : boolean, optional, default True This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. max_iter : integer, optional Maximum numbers of iterations to perform, therefore maximum features to include. 10% of ``n_features`` but at least 5 if available. cv : int, cross-validation generator or an iterable, optional Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 3-fold cross-validation, - integer, to specify the number of folds. - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. For integer/None inputs, :class:`KFold` is used. Refer :ref:`User Guide ` for the various cross-validation strategies that can be used here. .. versionchanged:: 0.20 ``cv`` default value if None will change from 3-fold to 5-fold in v0.22. n_jobs : int or None, optional (default=None) Number of CPUs to use during the cross validation. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. verbose : boolean or integer, optional Sets the verbosity amount Attributes ---------- intercept_ : float or array, shape (n_targets,) Independent term in decision function. coef_ : array, shape (n_features,) or (n_targets, n_features) Parameter vector (w in the problem formulation). n_nonzero_coefs_ : int Estimated number of non-zero coefficients giving the best mean squared error over the cross-validation folds. n_iter_ : int or array-like Number of active features across every target for the model refit with the best hyperparameters got by cross-validating across all folds. Examples -------- >>> from sklearn.linear_model import OrthogonalMatchingPursuitCV >>> from sklearn.datasets import make_regression >>> X, y = make_regression(n_features=100, n_informative=10, ... noise=4, random_state=0) >>> reg = OrthogonalMatchingPursuitCV(cv=5).fit(X, y) >>> reg.score(X, y) # doctest: +ELLIPSIS 0.9991... >>> reg.n_nonzero_coefs_ 10 >>> reg.predict(X[:1,]) array([-78.3854...]) See also -------- orthogonal_mp orthogonal_mp_gram lars_path Lars LassoLars OrthogonalMatchingPursuit LarsCV LassoLarsCV decomposition.sparse_encode TNr)FcCs.||_||_||_||_||_||_||_dS)N)rrirjrxcvn_jobsverbose)rkrrirjrxr{r|r}r?r?r@rlYsz$OrthogonalMatchingPursuitCV.__init__cstddd\tdddtjdd}js\tttdjdd jdnjt j j d fd d | D}td d |Dt fdd|D}t |jddd}|_t|jjd}||j_|j_|j_S)axFit the model using X, y as training data. Parameters ---------- X : array-like, shape [n_samples, n_features] Training data. y : array-like, shape [n_samples] Target values. Will be cast to X's dtype if necessary Returns ------- self : object returns an instance of self. Tr)rmZensure_min_featuresZ estimatorF)rZforce_all_finite)Z classifierg?r)r|r}c 3sB|]:\}}tt||||jjjVqdS)N)rryrrirj).0ZtrainZtest)r-rxrkr.r?r@ {sz2OrthogonalMatchingPursuitCV.fit..css|]}|jdVqdS)rN)r$)rfoldr?r?r@rscs$g|]}|ddjddqS)Nrr)rO)rw)rr)min_early_stopr?r@ sz3OrthogonalMatchingPursuitCV.fit..r)rO)r/rirj)r r r r{rxminrQrRr$rr|r}splitrZarrayZargminrwrnrhrirjrqrpZ intercept_ro)rkr-r.r{Zcv_pathsZ mse_foldsZbest_n_nonzero_coefsZompr?)r-rxrrkr.r@rqcs,  *   zOrthogonalMatchingPursuitCV.fit)TTTNr)NF)rrrsrtrurlrqr?r?r?r@rzsd rz)NTF)NNTTF)NNFTFF)NNNTTFF)TTTrv)rur(ZmathrZnumpyrZscipyrZscipy.linalg.lapackrbaserrr Zutilsr r r Zmodel_selectionr Z utils._joblibrrr*rArKrfrUrhryrzr?r?r?r@s:      p }     L