ó ‡ˆ\c@sjdZddlZddlmZddlZddlmZddlm Z ddl m Z m Z dd l m Z dd lmZmZmZdd lmZdd lmZmZd Zdeed„Zddeeed„Zddeeeed„Zdddeeeed„Zde e fd„ƒYZeeedd„Z de e fd„ƒYZ!dS(s'Orthogonal matching pursuit algorithms iÿÿÿÿN(tsqrt(tlinalg(tget_lapack_funcsi(t LinearModelt_pre_fiti(tRegressorMixin(tas_float_arrayt check_arrayt check_X_y(tcheck_cv(tParalleltdelayedsŒ Orthogonal matching pursuit ended prematurely due to linear dependence in the dictionary. The requested precision might not have been met. c Cs=|r|jdƒ}ntj|ƒ}tj|jƒj}tjd|fƒ\}}td|fƒ\} tj |j |ƒ} |} tj dƒ} d} tj |j dƒ}|d k rË|j dn|}tj ||fd|jƒ}|rtj|ƒ}nxîtr÷tjtjtj |j | ƒƒƒ}|| ksW| |d|krqtjttd dƒPn| dkrœtj |d d …d | …fj |d d …|fƒ|| d | …fR?tbeta((s7lib/python2.7/site-packages/sklearn/linear_model/omp.pyt _gram_omp‹st5$    &   %1 ,(     *c CsŒt|ddd|ƒ}t}|jdkrB|jddƒ}nt|ƒ}|jddkrjt}n|dkr¥|dkr¥ttd|jdƒdƒ}n|dk rÌ|dkrÌt dƒ‚n|dkró|dkrót d ƒ‚n|dkr!||jdkr!t d ƒ‚n|d krJ|jd|jdk}n|rßt j |j |ƒ}t j |ƒ}t j |j |ƒ} |dk r±t j|d d dƒ} nd} t|| ||| d|dtd|ƒS|rt j|jd|jd|jdfƒ} n#t j|jd|jdfƒ} g} xt|jdƒD]ì} t||dd…| f||d|d|ƒ}|r|\}}}}| dd…dd…dt|ƒ…f} xbt|j ƒD]/\}}||d | ||d | |f`. Parameters ---------- X : array, shape (n_samples, n_features) Input data. Columns are assumed to have unit norm. y : array, shape (n_samples,) or (n_samples, n_targets) Input targets n_nonzero_coefs : int Desired number of non-zero entries in the solution. If None (by default) this value is set to 10% of n_features. tol : float Maximum norm of the residual. If not None, overrides n_nonzero_coefs. precompute : {True, False, 'auto'}, Whether to perform precomputations. Improves performance when n_targets or n_samples is very large. copy_X : bool, optional Whether the design matrix X must be copied by the algorithm. A false value is only helpful if X is already Fortran-ordered, otherwise a copy is made anyway. return_path : bool, optional. Default: False Whether to return every value of the nonzero coefficients along the forward path. Useful for cross-validation. return_n_iter : bool, optional default False Whether or not to return the number of iterations. Returns ------- coef : array, shape (n_features,) or (n_features, n_targets) Coefficients of the OMP solution. If `return_path=True`, this contains the whole coefficient path. In this case its shape is (n_features, n_features) or (n_features, n_targets, n_features) and iterating over the last axis yields coefficients in increasing order of active features. n_iters : array-like or int Number of active features across every target. Returned only if `return_n_iter` is set to True. See also -------- OrthogonalMatchingPursuit orthogonal_mp_gram lars_path decomposition.sparse_encode Notes ----- Orthogonal matching pursuit was introduced in S. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Transactions on Signal Processing, Vol. 41, No. 12. (December 1993), pp. 3397-3415. (http://blanche.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf) This implementation is based on Rubinstein, R., Zibulevsky, M. and Elad, M., Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Technical Report - CS Technion, April 2008. http://www.cs.technion.ac.il/~ronrubin/Publications/KSVD-OMP-v2.pdf torderR Riiÿÿÿÿgš™™™™™¹?isEpsilon cannot be negatives$The number of atoms must be positives>The number of atoms cannot be more than the number of featurestautoitaxisRHRIR2NR1(RR+tndimtreshapeR R#R!tmaxtintt ValueErrorRRRRtsumtorthogonal_mp_gramtzerostrangeR@RCt enumeratetappendtsqueeze(R-R.R/R0t precomputeR1R2t return_n_itertGRFt norms_squaredtcoeftn_iterstktoutR?tidxR;tn_iterR7tx((s7lib/python2.7/site-packages/sklearn/linear_model/omp.pyt orthogonal_mp s`Q  #   0#+* c  CsSt|ddd|ƒ}tj|ƒ}|jdkrR|jddkrRt}n|jdkr•|dd…tjf}|dk r•|g}q•n|s¨|jj r·|j ƒ}n|dkrè|dkrèt dt |ƒƒ}n|dk r|dkrt dƒ‚n|dk r6|dkr6t d ƒ‚n|dkr]|dkr]t d ƒ‚n|dkrŠ|t |ƒkrŠt d ƒ‚n|r¾tjt |ƒ|jdt |ƒfƒ} n"tjt |ƒ|jdfƒ} g} x t|jdƒD] } t||dd…| f||dk r2|| nd|d |d td|ƒ} |rÙ| \} }}}| dd…dd…dt |ƒ…f} xbt|jƒD]/\}}||d | ||d | |f`. Parameters ---------- Gram : array, shape (n_features, n_features) Gram matrix of the input data: X.T * X Xy : array, shape (n_features,) or (n_features, n_targets) Input targets multiplied by X: X.T * y n_nonzero_coefs : int Desired number of non-zero entries in the solution. If None (by default) this value is set to 10% of n_features. tol : float Maximum norm of the residual. If not None, overrides n_nonzero_coefs. norms_squared : array-like, shape (n_targets,) Squared L2 norms of the lines of y. Required if tol is not None. copy_Gram : bool, optional Whether the gram matrix must be copied by the algorithm. A false value is only helpful if it is already Fortran-ordered, otherwise a copy is made anyway. copy_Xy : bool, optional Whether the covariance vector Xy must be copied by the algorithm. If False, it may be overwritten. return_path : bool, optional. Default: False Whether to return every value of the nonzero coefficients along the forward path. Useful for cross-validation. return_n_iter : bool, optional default False Whether or not to return the number of iterations. Returns ------- coef : array, shape (n_features,) or (n_features, n_targets) Coefficients of the OMP solution. If `return_path=True`, this contains the whole coefficient path. In this case its shape is (n_features, n_features) or (n_features, n_targets, n_features) and iterating over the last axis yields coefficients in increasing order of active features. n_iters : array-like or int Number of active features across every target. Returned only if `return_n_iter` is set to True. See also -------- OrthogonalMatchingPursuit orthogonal_mp lars_path decomposition.sparse_encode Notes ----- Orthogonal matching pursuit was introduced in G. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Transactions on Signal Processing, Vol. 41, No. 12. (December 1993), pp. 3397-3415. (http://blanche.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf) This implementation is based on Rubinstein, R., Zibulevsky, M. and Elad, M., Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Technical Report - CS Technion, April 2008. http://www.cs.technion.ac.il/~ronrubin/Publications/KSVD-OMP-v2.pdf RNR RiNgš™™™™™¹?sSGram OMP needs the precomputed norms in order to evaluate the error sum of squares.isEpsilon cannot be negatives$The number of atoms must be positives>The number of atoms cannot be more than the number of featuresRHRIR2(RRtasarrayRQR R#tnewaxisR!RARBRRTRCRURXRYRMR+RZRR[R\(RERFR/R0R`RHRIR2R^RaRbRcRdR?ReR;RfR7Rg((s7lib/python2.7/site-packages/sklearn/linear_model/omp.pyRW—sVN"  ."  +* tOrthogonalMatchingPursuitcBs/eZdZddeedd„Zd„ZRS(s Orthogonal Matching Pursuit model (OMP) Read more in the :ref:`User Guide `. Parameters ---------- n_nonzero_coefs : int, optional Desired number of non-zero entries in the solution. If None (by default) this value is set to 10% of n_features. tol : float, optional Maximum norm of the residual. If not None, overrides n_nonzero_coefs. fit_intercept : boolean, optional whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). normalize : boolean, optional, default True This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. precompute : {True, False, 'auto'}, default 'auto' Whether to use a precomputed Gram and Xy matrix to speed up calculations. Improves performance when `n_targets` or `n_samples` is very large. Note that if you already have such matrices, you can pass them directly to the fit method. Attributes ---------- coef_ : array, shape (n_features,) or (n_targets, n_features) parameter vector (w in the formula) intercept_ : float or array, shape (n_targets,) independent term in decision function. n_iter_ : int or array-like Number of active features across every target. Examples -------- >>> from sklearn.linear_model import OrthogonalMatchingPursuit >>> from sklearn.datasets import make_regression >>> X, y = make_regression(noise=4, random_state=0) >>> reg = OrthogonalMatchingPursuit().fit(X, y) >>> reg.score(X, y) # doctest: +ELLIPSIS 0.9991... >>> reg.predict(X[:1,]) array([-78.3854...]) Notes ----- Orthogonal matching pursuit was introduced in G. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Transactions on Signal Processing, Vol. 41, No. 12. (December 1993), pp. 3397-3415. (http://blanche.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf) This implementation is based on Rubinstein, R., Zibulevsky, M. and Elad, M., Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Technical Report - CS Technion, April 2008. http://www.cs.technion.ac.il/~ronrubin/Publications/KSVD-OMP-v2.pdf See also -------- orthogonal_mp orthogonal_mp_gram lars_path Lars LassoLars decomposition.sparse_encode OrthogonalMatchingPursuitCV ROcCs1||_||_||_||_||_dS(N(R/R0t fit_interceptt normalizeR](tselfR/R0RlRmR]((s7lib/python2.7/site-packages/sklearn/linear_model/omp.pyt__init__ks     c Cs¼t||dtdtƒ\}}|jd}t||d|j|j|jdtƒ\}}}}}}}|jdkr˜|dd…t j f}n|j dkrÕ|j dkrÕt td|ƒdƒ|_n |j |_|tkr&t|||j|j dtdtd tƒ\} |_ns|j dk rNt j|d d d ƒnd} t|d |d|jd|j d| dtdtd tƒ\} |_| j|_|j|||ƒ|S(s”Fit the model using X, y as training data. Parameters ---------- X : array-like, shape (n_samples, n_features) Training data. y : array-like, shape (n_samples,) or (n_samples, n_targets) Target values. Will be cast to X's dtype if necessary Returns ------- self : object returns an instance of self. t multi_outputt y_numericiRNgš™™™™™¹?R]R1R^iRPiRFR/R0R`RHRI(RR#R RR!R]RmRlRQRRjR/R0RSRTtn_nonzero_coefs_R+Rhtn_iter_RVRWRtcoef_t_set_intercept( RnR-R.t n_featurestX_offsetty_offsettX_scaleRERFRttnorms_sq((s7lib/python2.7/site-packages/sklearn/linear_model/omp.pytfitss.! '   .  N(t__name__t __module__t__doc__R!R#RoR{(((s7lib/python2.7/site-packages/sklearn/linear_model/omp.pyRksL idc Cs®|r9|jƒ}|jƒ}|jƒ}|jƒ}n|r²|jddƒ}||8}||8}|jddƒ} t|dtƒ}|| 8}t|dtƒ}|| 8}n|r tjtj|dddƒƒ} tj| ƒ} |dd…| fc| | :`. Parameters ---------- copy : bool, optional Whether the design matrix X must be copied by the algorithm. A false value is only helpful if X is already Fortran-ordered, otherwise a copy is made anyway. fit_intercept : boolean, optional whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered). normalize : boolean, optional, default True This parameter is ignored when ``fit_intercept`` is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use :class:`sklearn.preprocessing.StandardScaler` before calling ``fit`` on an estimator with ``normalize=False``. max_iter : integer, optional Maximum numbers of iterations to perform, therefore maximum features to include. 10% of ``n_features`` but at least 5 if available. cv : int, cross-validation generator or an iterable, optional Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 3-fold cross-validation, - integer, to specify the number of folds. - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. For integer/None inputs, :class:`KFold` is used. Refer :ref:`User Guide ` for the various cross-validation strategies that can be used here. .. versionchanged:: 0.20 ``cv`` default value if None will change from 3-fold to 5-fold in v0.22. n_jobs : int or None, optional (default=None) Number of CPUs to use during the cross validation. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. verbose : boolean or integer, optional Sets the verbosity amount Attributes ---------- intercept_ : float or array, shape (n_targets,) Independent term in decision function. coef_ : array, shape (n_features,) or (n_targets, n_features) Parameter vector (w in the problem formulation). n_nonzero_coefs_ : int Estimated number of non-zero coefficients giving the best mean squared error over the cross-validation folds. n_iter_ : int or array-like Number of active features across every target for the model refit with the best hyperparameters got by cross-validating across all folds. Examples -------- >>> from sklearn.linear_model import OrthogonalMatchingPursuitCV >>> from sklearn.datasets import make_regression >>> X, y = make_regression(n_features=100, n_informative=10, ... noise=4, random_state=0) >>> reg = OrthogonalMatchingPursuitCV(cv=5).fit(X, y) >>> reg.score(X, y) # doctest: +ELLIPSIS 0.9991... >>> reg.n_nonzero_coefs_ 10 >>> reg.predict(X[:1,]) array([-78.3854...]) See also -------- orthogonal_mp orthogonal_mp_gram lars_path Lars LassoLars OrthogonalMatchingPursuit LarsCV LassoLarsCV decomposition.sparse_encode R'cCsC||_||_||_||_||_||_||_dS(N(RRlRmR…tcvtn_jobstverbose(RnRRlRmR…RŒRRŽ((s7lib/python2.7/site-packages/sklearn/linear_model/omp.pyRoYs      c s¦tˆˆdtdddˆƒ\‰‰tˆdtdtƒ‰tˆjdtƒ}ˆjsttt dˆj d ƒd ƒˆj d ƒnˆj‰t d ˆj d ˆj ƒ‡‡‡‡fd †|jˆƒDƒƒ}td„|Dƒƒ}tjg|D] }|| djdd ƒ^qüƒ}tj|jddƒƒd }|ˆ_td|dˆjdˆjƒ} | jˆˆƒ| jˆ_| jˆ_| jˆ_ˆS(sxFit the model using X, y as training data. Parameters ---------- X : array-like, shape [n_samples, n_features] Training data. y : array-like, shape [n_samples] Target values. Will be cast to X's dtype if necessary Returns ------- self : object returns an instance of self. Rqtensure_min_featuresit estimatorRtforce_all_finitet classifiergš™™™™™¹?iiRRŽc 3sU|]K\}}ttƒˆ|ˆ|ˆ|ˆ|ˆjˆjˆjˆƒVqdS(N(R RŠRRlRm(t.0ttrainttest(R-R…RnR.(s7lib/python2.7/site-packages/sklearn/linear_model/omp.pys {scss|]}|jdVqdS(iN(R (R“tfold((s7lib/python2.7/site-packages/sklearn/linear_model/omp.pys €sRPiR/RlRm(RR#RR+R RŒR…tminRSRTR R RRŽtsplitRtarrayRtargminRrRkRlRmR{Rtt intercept_Rs( RnR-R.RŒtcv_pathstmin_early_stopR–t mse_foldstbest_n_nonzero_coefstomp((R-R…RnR.s7lib/python2.7/site-packages/sklearn/linear_model/omp.pyR{cs,9  -       N(R|R}R~R#R!R+RoR{(((s7lib/python2.7/site-packages/sklearn/linear_model/omp.pyR‹ôsd  ("R~R&tmathRtnumpyRtscipyRtscipy.linalg.lapackRtbaseRRRtutilsRRRtmodel_selectionR t utils._joblibR R R(R!R#R+R@RMRhRWRkRŠR‹(((s7lib/python2.7/site-packages/sklearn/linear_model/omp.pyts4   p}  Œ „‰L