ó ‡ˆ\c@srdZddlmZddlZddlZddlmZddlm Z ddl m Z ddl m Z dd lmZmZmZdd lmZdd lmZmZmZdd lmZdd lmZmZddlmZddlmZddlm Z ddgZ!e"d„Z#d„Z$e"d„Z%deeefd„ƒYZ&deefd„ƒYZ'dS(sB Linear Discriminant Analysis and Quadratic Discriminant Analysis iÿÿÿÿ(tprint_functionNi(t deprecated(tlinalg(t string_types(txrange(t BaseEstimatortTransformerMixintClassifierMixin(tLinearClassifierMixin(t ledoit_wolftempirical_covariancetshrunk_covariance(t unique_labels(t check_arrayt check_X_y(tcheck_is_fitted(tcheck_classification_targets(tStandardScalertLinearDiscriminantAnalysistQuadraticDiscriminantAnalysiscCs/|dkrdn|}t|tƒrÂ|dkr˜tƒ}|j|ƒ}t|ƒd}|jdd…tjf||jtjdd…f}q+|dkr³t |ƒ}q+t dƒ‚nit|t ƒsàt|t ƒr|dksø|dkrt dƒ‚nt t |ƒ|ƒ}n tdƒ‚|S( s!Estimate covariance matrix (using optional shrinkage). Parameters ---------- X : array-like, shape (n_samples, n_features) Input data. shrinkage : string or float, optional Shrinkage parameter, possible values: - None or 'empirical': no shrinkage (default). - 'auto': automatic shrinkage using the Ledoit-Wolf lemma. - float between 0 and 1: fixed shrinkage parameter. Returns ------- s : array, shape (n_features, n_features) Estimated covariance matrix. t empiricaltautoiNsunknown shrinkage parameteris+shrinkage parameter must be between 0 and 1s'shrinkage must be of string or int type(tNonet isinstanceRRt fit_transformR tscale_tnptnewaxisR t ValueErrortfloattintR t TypeError(tXt shrinkagetscts((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyt_cov!s   =  cCsƒtj|dtƒ\}}tj|ƒ}tjdt|ƒ|jdfƒ}tjj|||ƒ||dd…df:}|S(s;Compute class means. Parameters ---------- X : array-like, shape (n_samples, n_features) Input data. y : array-like, shape (n_samples,) or (n_samples, n_targets) Target values. Returns ------- means : array-like, shape (n_classes, n_features) Class means. treturn_inversetshapeiN( RtuniquetTruetbincounttzerostlenR&taddtatR(R tytclassestcnttmeans((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyt _class_meansIs %c Cs–tj|ƒ}tjd|jd|jdfƒ}xZt|ƒD]L\}}|||kdd…f}|||tjt||ƒƒ7}qBW|S(s›Compute class covariance matrix. Parameters ---------- X : array-like, shape (n_samples, n_features) Input data. y : array-like, shape (n_samples,) or (n_samples, n_targets) Target values. priors : array-like, shape (n_classes,) Class priors. shrinkage : string or float, optional Shrinkage parameter, possible values: - None: no shrinkage (default). - 'auto': automatic shrinkage using the Ledoit-Wolf lemma. - float between 0 and 1: fixed shrinkage parameter. Returns ------- cov : array-like, shape (n_features, n_features) Class covariance matrix. R&iN(RR'R*R&t enumeratet atleast_2dR$( R R.tpriorsR!R/tcovtidxtgrouptXg((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyt _class_covas &(cBsheZdZdd d d edd„Zd„Zd„Zd„Zd„Z d„Z d „Z d „Z RS( sâLinear Discriminant Analysis A classifier with a linear decision boundary, generated by fitting class conditional densities to the data and using Bayes' rule. The model fits a Gaussian density to each class, assuming that all classes share the same covariance matrix. The fitted model can also be used to reduce the dimensionality of the input by projecting it to the most discriminative directions. .. versionadded:: 0.17 *LinearDiscriminantAnalysis*. Read more in the :ref:`User Guide `. Parameters ---------- solver : string, optional Solver to use, possible values: - 'svd': Singular value decomposition (default). Does not compute the covariance matrix, therefore this solver is recommended for data with a large number of features. - 'lsqr': Least squares solution, can be combined with shrinkage. - 'eigen': Eigenvalue decomposition, can be combined with shrinkage. shrinkage : string or float, optional Shrinkage parameter, possible values: - None: no shrinkage (default). - 'auto': automatic shrinkage using the Ledoit-Wolf lemma. - float between 0 and 1: fixed shrinkage parameter. Note that shrinkage works only with 'lsqr' and 'eigen' solvers. priors : array, optional, shape (n_classes,) Class priors. n_components : int, optional Number of components (< n_classes - 1) for dimensionality reduction. store_covariance : bool, optional Additionally compute class covariance matrix (default False), used only in 'svd' solver. .. versionadded:: 0.17 tol : float, optional, (default 1.0e-4) Threshold used for rank estimation in SVD solver. .. versionadded:: 0.17 Attributes ---------- coef_ : array, shape (n_features,) or (n_classes, n_features) Weight vector(s). intercept_ : array, shape (n_features,) Intercept term. covariance_ : array-like, shape (n_features, n_features) Covariance matrix (shared by all classes). explained_variance_ratio_ : array, shape (n_components,) Percentage of variance explained by each of the selected components. If ``n_components`` is not set then all components are stored and the sum of explained variances is equal to 1.0. Only available when eigen or svd solver is used. means_ : array-like, shape (n_classes, n_features) Class means. priors_ : array-like, shape (n_classes,) Class priors (sum to 1). scalings_ : array-like, shape (rank, n_classes - 1) Scaling of the features in the space spanned by the class centroids. xbar_ : array-like, shape (n_features,) Overall mean. classes_ : array-like, shape (n_classes,) Unique class labels. See also -------- sklearn.discriminant_analysis.QuadraticDiscriminantAnalysis: Quadratic Discriminant Analysis Notes ----- The default solver is 'svd'. It can perform both classification and transform, and it does not rely on the calculation of the covariance matrix. This can be an advantage in situations where the number of features is large. However, the 'svd' solver cannot be used with shrinkage. The 'lsqr' solver is an efficient algorithm that only works for classification. It supports shrinkage. The 'eigen' solver is based on the optimization of the between class scatter to within class scatter ratio. It can be used for both classification and transform, and it supports shrinkage. However, the 'eigen' solver needs to compute the covariance matrix, so it might not be suitable for situations with a high number of features. Examples -------- >>> import numpy as np >>> from sklearn.discriminant_analysis import LinearDiscriminantAnalysis >>> X = np.array([[-1, -1], [-2, -1], [-3, -2], [1, 1], [2, 1], [3, 2]]) >>> y = np.array([1, 1, 1, 2, 2, 2]) >>> clf = LinearDiscriminantAnalysis() >>> clf.fit(X, y) LinearDiscriminantAnalysis(n_components=None, priors=None, shrinkage=None, solver='svd', store_covariance=False, tol=0.0001) >>> print(clf.predict([[-0.8, -1]])) [1] tsvdg-Cëâ6?cCs:||_||_||_||_||_||_dS(N(tsolverR!R5t n_componentststore_covariancettol(tselfR<R!R5R=R>R?((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyt__init__ús      cCs‘t||ƒ|_t|||j|ƒ|_tj|j|jjƒdj|_dt j t j |j|jjƒƒt j |jƒ|_ dS(s¶Least squares solver. The least squares solver computes a straightforward solution of the optimal decision rule based directly on the discriminant functions. It can only be used for classification (with optional shrinkage), because estimation of eigenvectors is not performed. Therefore, dimensionality reduction with the transform is not supported. Parameters ---------- X : array-like, shape (n_samples, n_features) Training data. y : array-like, shape (n_samples,) or (n_samples, n_classes) Target values. shrinkage : string or float, optional Shrinkage parameter, possible values: - None: no shrinkage (default). - 'auto': automatic shrinkage using the Ledoit-Wolf lemma. - float between 0 and 1: fixed shrinkage parameter. Notes ----- This solver is based on [1]_, section 2.6.2, pp. 39-41. References ---------- .. [1] R. O. Duda, P. E. Hart, D. G. Stork. Pattern Classification (Second Edition). John Wiley & Sons, Inc., New York, 2001. ISBN 0-471-05669-3. igà¿N(R2tmeans_R:tpriors_t covariance_RtlstsqtTtcoef_Rtdiagtdottlogt intercept_(R@R R.R!((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyt _solve_lsqrs !%%c CsNt||ƒ|_t|||j|ƒ|_|j}t||ƒ}||}tj||ƒ\}}tj |tj |ƒƒddd…|j |_ |dd…tj |ƒddd…f}|tjj|ddƒ:}||_tj|j|ƒj|jƒ|_dtjtj|j|jjƒƒtj|jƒ|_dS(s]Eigenvalue solver. The eigenvalue solver computes the optimal solution of the Rayleigh coefficient (basically the ratio of between class scatter to within class scatter). This solver supports both classification and dimensionality reduction (with optional shrinkage). Parameters ---------- X : array-like, shape (n_samples, n_features) Training data. y : array-like, shape (n_samples,) or (n_samples, n_targets) Target values. shrinkage : string or float, optional Shrinkage parameter, possible values: - None: no shrinkage (default). - 'auto': automatic shrinkage using the Ledoit-Wolf lemma. - float between 0 and 1: fixed shrinkage constant. Notes ----- This solver is based on [1]_, section 3.8.3, pp. 121-124. References ---------- .. [1] R. O. Duda, P. E. Hart, D. G. Stork. Pattern Classification (Second Edition). John Wiley & Sons, Inc., New York, 2001. ISBN 0-471-05669-3. Niÿÿÿÿtaxisigà¿(R2RBR:RCRDR$RteighRtsorttsumt_max_componentstexplained_variance_ratio_targsorttnormt scalings_RIRFRGRHRJRK( R@R R.R!tSwtSttSbtevalstevecs((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyt _solve_eigen*s   , $%cCsí|j\}}t|jƒ}t||ƒ|_|jrTt|||jƒ|_ng}xQt |jƒD]@\}}|||kdd…f} |j | |j|ƒqjWt j |j|jƒ|_ t j|ddƒ}|jddƒ} d| | dkR:RCRDR3tappendRRItxbar_t concatenatetstdtsqrtRR;tFalseRPR?twarningstwarnRFRQRRRURJRKRG(R@R R.t n_samplest n_featurest n_classestXcR7R8R9RatfactUtStVtranktscalingst_tcoef((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyt _solve_svd\s@     .cCsµt||ddd|ƒ\}}t|ƒ|_|j\}}t|jƒ}||kritdƒ‚n|jdkr¸tj |dt ƒ\}}tj |ƒt t|ƒƒ|_ ntj|jƒ|_ |j dkjƒrñtdƒ‚ntj|j jƒdƒs8tjd tƒ|j |j jƒ|_ n|jdkr`t|jƒd |_n"tt|jƒd |jƒ|_|jd krÂ|jdk r¯td ƒ‚n|j||ƒnn|jd krí|j||d|jƒnC|jdkr|j||d|jƒntdj|jƒƒ‚|jjdkr±tj |j!d dd…f|j!ddd…fddƒ|_!tj |j"d |j"ddd ƒ|_"n|S(sîFit LinearDiscriminantAnalysis model according to the given training data and parameters. .. versionchanged:: 0.19 *store_covariance* has been moved to main constructor. .. versionchanged:: 0.19 *tol* has been moved to main constructor. Parameters ---------- X : array-like, shape (n_samples, n_features) Training data. y : array, shape (n_samples,) Target values. tensure_min_samplesit estimators>The number of samples must be more than the number of classes.R%ispriors must be non-negativegð?s)The priors do not sum to 1. RenormalizingiR;sshrinkage not supportedtlsqrR!teigensAunknown solver {} (valid solvers are 'svd', 'lsqr', and 'eigen').Ntndmin(#RR R]R&R+RR5RRR'R(R)RRCtasarraytanytiscloseRPRdRet UserWarningR=RQtminR<R!tNotImplementedErrorRrRLR[tformattsizetarrayRGRK(R@R R.RfRpRhty_t((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pytfit›sF! %  BcCsµ|jdkrtdƒ‚nt|ddgdtƒt|ƒ}|jdkrqtj||j|jƒ}n'|jdkr˜tj||jƒ}n|dd…d|j …fS( sProject data to maximize class separation. Parameters ---------- X : array-like, shape (n_samples, n_features) Input data. Returns ------- X_new : array, shape (n_samples, n_components) Transformed data. RusCtransform not implemented for 'lsqr' solver (use 'svd' or 'eigen').R_RUt all_or_anyR;RvN( R<R}RRyR RRIR_RURQ(R@R tX_new((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyt transformÛs  cCs£|j|ƒ}|d9}tj||ƒ|d7}tj||ƒt|jƒdkrotjd||gƒS||jddƒj|j ddfƒ:}|SdS(s Estimate probability. Parameters ---------- X : array-like, shape (n_samples, n_features) Input data. Returns ------- C : array, shape (n_samples, n_classes) Estimated probabilities. iÿÿÿÿiiRMiN( tdecision_functionRtexpt reciprocalR+R]t column_stackRPtreshapeR&(R@R tprob((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyt predict_probaõs   ,cCstj|j|ƒƒS(sEstimate log probability. Parameters ---------- X : array-like, shape (n_samples, n_features) Input data. Returns ------- C : array, shape (n_samples, n_classes) Estimated log probabilities. (RRJRŒ(R@R ((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pytpredict_log_probas N( t__name__t __module__t__doc__RRcRARLR[RrR‚R…RŒR(((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyR‚sv  ' 2 ? @  cBsweZdZd dedd d„Zeedƒd„ƒƒZd„Z d„Z d„Z d „Z d „Z d „ZRS( sà Quadratic Discriminant Analysis A classifier with a quadratic decision boundary, generated by fitting class conditional densities to the data and using Bayes' rule. The model fits a Gaussian density to each class. .. versionadded:: 0.17 *QuadraticDiscriminantAnalysis* Read more in the :ref:`User Guide `. Parameters ---------- priors : array, optional, shape = [n_classes] Priors on classes reg_param : float, optional Regularizes the covariance estimate as ``(1-reg_param)*Sigma + reg_param*np.eye(n_features)`` store_covariance : boolean If True the covariance matrices are computed and stored in the `self.covariance_` attribute. .. versionadded:: 0.17 tol : float, optional, default 1.0e-4 Threshold used for rank estimation. .. versionadded:: 0.17 store_covariances : boolean Deprecated, use `store_covariance`. Attributes ---------- covariance_ : list of array-like, shape = [n_features, n_features] Covariance matrices of each class. means_ : array-like, shape = [n_classes, n_features] Class means. priors_ : array-like, shape = [n_classes] Class priors (sum to 1). rotations_ : list of arrays For each class k an array of shape [n_features, n_k], with ``n_k = min(n_features, number of elements in class k)`` It is the rotation of the Gaussian distribution, i.e. its principal axis. scalings_ : list of arrays For each class k an array of shape [n_k]. It contains the scaling of the Gaussian distributions along its principal axes, i.e. the variance in the rotated coordinate system. Examples -------- >>> from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis >>> import numpy as np >>> X = np.array([[-1, -1], [-2, -1], [-3, -2], [1, 1], [2, 1], [3, 2]]) >>> y = np.array([1, 1, 1, 2, 2, 2]) >>> clf = QuadraticDiscriminantAnalysis() >>> clf.fit(X, y) ... # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE QuadraticDiscriminantAnalysis(priors=None, reg_param=0.0, store_covariance=False, store_covariances=None, tol=0.0001) >>> print(clf.predict([[-0.8, -1]])) [1] See also -------- sklearn.discriminant_analysis.LinearDiscriminantAnalysis: Linear Discriminant Analysis gg-Cëâ6?cCsL|dk rtj|ƒnd|_||_||_||_||_dS(N(RRRxR5t reg_paramtstore_covariancesR>R?(R@R5R‘R>R?R’((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyRAns $   srAttribute ``covariances_`` was deprecated in version 0.19 and will be removed in 0.21. Use ``covariance_`` insteadcCs|jS(N(RD(R@((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyt covariances_vscCs«t||ƒ\}}t|ƒtj|dtƒ\|_}|j\}}t|jƒ}|dkrztd|ƒ‚n|j dkr¨tj |ƒt |ƒ|_ n |j |_ d}|jpÉ|j}|jrètjdtƒn|r÷g}ng}g} g} x\t|ƒD]N} ||| kdd…f} | jdƒ} |j| ƒt| ƒdkr†tdt|j| ƒƒ‚n| | }tjj|d tƒ\}}}tj||jkƒ}||kråtjd ƒn|dt| ƒd}d|j||j}|js$|rG|jtj||j|ƒƒn| j|ƒ| j|jƒqW|jsw|rƒ||_ntj |ƒ|_!| |_"| |_#|S( siFit the model according to the given training data and parameters. .. versionchanged:: 0.19 ``store_covariances`` has been moved to main constructor as ``store_covariance`` .. versionchanged:: 0.19 ``tol`` has been moved to main constructor. Parameters ---------- X : array-like, shape = [n_samples, n_features] Training vector, where n_samples is the number of samples and n_features is the number of features. y : array, shape = [n_samples] Target values (integers) R%is>The number of classes has to be greater than one; got %d classs`'store_covariances' was renamed to store_covariance in version 0.19 and will be removed in 0.21.Niis;y has only 1 sample in class %s, covariance is ill defined.R\sVariables are collinear($RRRR'R(R]R&R+RR5RR)RRCR>R’RdRetDeprecationWarningRtmeanR^tstrRR;RcRPR?R‘RIRFRDRxRBRUt rotations_(R@R R.RfRgRhR6R>R1Rot rotationstindR9tmeangtXgcRkRltVtRntS2((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyR‚}sZ          ! #    c Cst|dƒt|ƒ}g}xtt|jƒƒD]h}|j|}|j|}||j|}tj |||dƒ}|j tj |ddƒƒq5Wtj |ƒj }tjg|jD]}tj tj|ƒƒ^qÃ} d|| tj|jƒS(NR]gà¿ii(RR trangeR+R]R—RURBRRIR^RPR€RFRxRJRC( R@R tnorm2titRRltXmtX2R#tu((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyt_decision_functionÃs    !7cCsP|j|ƒ}t|jƒdkrL|dd…df|dd…dfS|S(sðApply decision function to an array of samples. Parameters ---------- X : array-like, shape = [n_samples, n_features] Array of samples (test vectors). Returns ------- C : array, shape = [n_samples, n_classes] or [n_samples,] Decision function values related to each class, per sample. In the two-class case, the shape is [n_samples,], giving the log likelihood ratio of the positive class. iNii(R¥R+R](R@R tdec_func((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyR†Òs(cCs.|j|ƒ}|jj|jdƒƒ}|S(s&Perform classification on an array of test vectors X. The predicted class C for each sample in X is returned. Parameters ---------- X : array-like, shape = [n_samples, n_features] Returns ------- C : array, shape = [n_samples] i(R¥R]ttaketargmax(R@R tdty_pred((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pytpredictçs cCsh|j|ƒ}tj||jddƒdd…tjfƒ}||jddƒdd…tjfS(sXReturn posterior probabilities of classification. Parameters ---------- X : array-like, shape = [n_samples, n_features] Array of samples/test vectors. Returns ------- C : array, shape = [n_samples, n_classes] Posterior probabilities of classification per class. RMiN(R¥RR‡tmaxRRP(R@R tvaluest likelihood((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyRŒøs 2cCs|j|ƒ}tj|ƒS(s\Return posterior probabilities of classification. Parameters ---------- X : array-like, shape = [n_samples, n_features] Array of samples/test vectors. Returns ------- C : array, shape = [n_samples, n_classes] Posterior log-probabilities of classification per class. (RŒRRJ(R@R tprobas_((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyR sN(RŽRRRRcRAtpropertyRR“R‚R¥R†R«RŒR(((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyRsN  F    ((Rt __future__RRdtnumpyRtutilsRtscipyRt externals.sixRtexternals.six.movesRtbaseRRRtlinear_model.baseRt covarianceR R R tutils.multiclassR R Rtutils.validationRRt preprocessingRt__all__RR$R2R:RR(((s<lib/python2.7/site-packages/sklearn/discriminant_analysis.pyts.     (  ! ÿœ