ó ‡ˆ\c@sœdZddlZddlZddlmZmZddlmZddl m Z ddl m Z ddl mZmZmZmZmZdd lmZmZdd lmZd d l mZd „Zd„Zd„Zd„Zd„Zed„Zde efd„ƒYZ de fd„ƒYZ!de fd„ƒYZ"de fd„ƒYZ#de fd„ƒYZ$de fd„ƒYZ%de fd „ƒYZ&dS(!sUnivariate features selection.iÿÿÿÿN(tspecialtstats(tissparsei(t BaseEstimator(tLabelBinarizer(tas_float_arrayt check_arrayt check_X_ytsafe_sqrt safe_mask(tsafe_sparse_dott row_norms(tcheck_is_fittedi(t SelectorMixincCs8t|dtƒ}tj|jƒj|tj|ƒ<|S(s– Fixes Issue #1240: NaNs can't be properly compared, so change them to the smallest value of scores's dtype. -inf seems to be unreliable. tcopy(RtTruetnptfinfotdtypetmintisnan(tscores((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyt _clean_nanss"cGs t|ƒ}g|D]}t|ƒ^q}tjg|D]}|jd^q8ƒ}tj|ƒ}td„|Dƒƒ}g|D]!}tj|jddƒƒ^q€}t|ƒd}g|D]}|d^q¾} ||t|ƒ} d} x0t|ƒD]"\} } | | | || 7} qûW| |t|ƒ8} | | }|d}||}| t|ƒ}|t|ƒ}tj |dkƒd}tj |ƒdj |j krË|j rËt j d|tƒn||}tj|ƒjƒ}tj|||ƒ}||fS(s*Performs a 1-way ANOVA. The one-way ANOVA tests the null hypothesis that 2 or more groups have the same population mean. The test is applied to samples from two or more groups, possibly with differing sizes. Read more in the :ref:`User Guide `. Parameters ---------- *args : array_like, sparse matrices sample1, sample2... The sample measurements should be given as arguments. Returns ------- F-value : float The computed F-value of the test. p-value : float The associated p-value from the F-distribution. Notes ----- The ANOVA test has important assumptions that must be satisfied in order for the associated p-value to be valid. 1. The samples are independent 2. Each sample is from a normally distributed population 3. The population standard deviations of the groups are all equal. This property is known as homoscedasticity. If these assumptions are not true for a given set of data, it may still be possible to use the Kruskal-Wallis H-test (`scipy.stats.kruskal`_) although with some loss of power. The algorithm is from Heiman[2], pp.394-7. See ``scipy.stats.f_oneway`` that should give the same results while being less efficient. References ---------- .. [1] Lowry, Richard. "Concepts and Applications of Inferential Statistics". Chapter 14. http://faculty.vassar.edu/lowry/ch14pt1.html .. [2] Heiman, G.W. Research Methods in Statistics. 2002. icss'|]}t|ƒjddƒVqdS(taxisiN(Rtsum(t.0ta((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pys asRigisFeatures %s are constant.(tlenRRtarraytshapeRtasarraytfloatt enumeratetwheretnonzerotsizetwarningstwarnt UserWarningtravelRtfdtrc(targst n_classesRtn_samples_per_classt n_samplest ss_alldatat sums_argstsquare_of_sums_alldatatstsquare_of_sums_argstsstottssbntkt_tsswntdfbntdfwntmsbtmswtconstant_features_idxtftprob((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pytf_oneway*s43 ).   (   cCs`t||dddgƒ\}}gtj|ƒD]}|t|||kƒ^q1}t|ŒS(s¯Compute the ANOVA F-value for the provided sample. Read more in the :ref:`User Guide `. Parameters ---------- X : {array-like, sparse matrix} shape = [n_samples, n_features] The set of regressors that will be tested sequentially. y : array of shape(n_samples) The data matrix. Returns ------- F : array, shape = [n_features,] The set of F values. pval : array, shape = [n_features,] The set of p-values. See also -------- chi2: Chi-squared stats of non-negative features for classification tasks. f_regression: F-value between label/feature for regression tasks. tcsrtcsctcoo(RRtuniqueR R>(tXtyR4R)((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyt f_classifzs!5cCstj|dtjƒ}t|ƒ}|}||8}|dC}tjddƒ||:}WdQX|jddƒ}|tj|d|ƒfS( sŽFast replacement for scipy.stats.chisquare. Version from https://github.com/scipy/scipy/pull/2525 with additional optimizations. RitinvalidtignoreNRii(RRtfloat64RterrstateRRtchdtrc(tf_obstf_expR4tchisq((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyt _chisquare™s   cCsÿt|ddƒ}tjt|ƒr-|jn|dkƒrKtdƒ‚ntƒj|ƒ}|jddkrtj d||ddƒ}nt |j |ƒ}|j ddƒj ddƒ}|jddƒj ddƒ}tj|j |ƒ}t||ƒS(s0Compute chi-squared stats between each non-negative feature and class. This score can be used to select the n_features features with the highest values for the test chi-squared statistic from X, which must contain only non-negative features such as booleans or frequencies (e.g., term counts in document classification), relative to the classes. Recall that the chi-square test measures dependence between stochastic variables, so using this function "weeds out" the features that are the most likely to be independent of class and therefore irrelevant for classification. Read more in the :ref:`User Guide `. Parameters ---------- X : {array-like, sparse matrix}, shape = (n_samples, n_features_in) Sample vectors. y : array-like, shape = (n_samples,) Target vector (class labels). Returns ------- chi2 : array, shape = (n_features,) chi2 statistics of each feature. pval : array, shape = (n_features,) p-values of each feature. Notes ----- Complexity of this algorithm is O(n_classes * n_features). See also -------- f_classif: ANOVA F-value between label/feature for classification tasks. f_regression: F-value between label/feature for regression tasks. t accept_sparseR?isInput X must be non-negative.iRiÿÿÿÿ(RRtanyRtdatat ValueErrorRt fit_transformRtappendR tTRtreshapetmeantdotRN(RCRDtYtobservedt feature_countt class_probtexpected((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pytchi2¬s**c CsIt||dddgdtjƒ\}}|jd}|r¶|tj|ƒ}t|ƒrw|jddƒjƒ}n|jddƒ}tjt|j dt ƒ||dƒ}nt|j ƒ}t ||ƒ}||:}|tj j |ƒ:}|j|rdnd }|dd |d|}tjj|d |ƒ} || fS( stUnivariate linear regression tests. Linear model for testing the individual effect of each of many regressors. This is a scoring function to be used in a feature selection procedure, not a free standing feature selection procedure. This is done in 2 steps: 1. The correlation between each regressor and the target is computed, that is, ((X[:, i] - mean(X[:, i])) * (y - mean_y)) / (std(X[:, i]) * std(y)). 2. It is converted to an F score then to a p-value. For more on usage see the :ref:`User Guide `. Parameters ---------- X : {array-like, sparse matrix} shape = (n_samples, n_features) The set of regressors that will be tested sequentially. y : array of shape(n_samples). The data matrix center : True, bool, If true, X and y will be centered. Returns ------- F : array, shape=(n_features,) F values of features. pval : array, shape=(n_features,) p-values of F-scores. See also -------- mutual_info_regression: Mutual information for a continuous target. f_classif: ANOVA F-value between label/feature for classification tasks. chi2: Chi-squared stats of non-negative features for classification tasks. SelectKBest: Select features based on the k highest scores. SelectFpr: Select features based on a false positive rate test. SelectFdr: Select features based on an estimated false discovery rate. SelectFwe: Select features based on family-wise error rate. SelectPercentile: Select features based on percentile of the highest scores. R?R@RARiRtsquaredii(RRRHRRWRtgetA1tsqrtR RURR tlinalgtnormR#RR<tsf( RCRDtcenterR,tX_meanstX_normstcorrtdegrees_of_freedomtFtpv((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyt f_regressionçs"0*   t _BaseFiltercBs)eZdZd„Zd„Zd„ZRS(sîInitialize the univariate feature selection. Parameters ---------- score_func : callable Function taking two arrays X and y, and returning a pair of arrays (scores, pvalues) or a single array with scores. cCs ||_dS(N(t score_func(tselfRn((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyt__init__BscCsät||ddgdtƒ\}}t|jƒsXtd|jt|jƒfƒ‚n|j||ƒ|j||ƒ}t|tt fƒr¹|\|_ |_ t j |j ƒ|_ n||_ d|_ t j |j ƒ|_ |S(sšRun score function on (X, y) and get the appropriate features. Parameters ---------- X : array-like, shape = [n_samples, n_features] The training input samples. y : array-like, shape = [n_samples] The target values (class labels in classification, real numbers in regression). Returns ------- self : object R?R@t multi_outputs<The score function should be a callable, %s (%s) was passed.N(RRtcallableRnt TypeErrorttypet _check_paramst isinstancetlistttupletscores_tpvalues_RRtNone(RoRCRDtscore_func_ret((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pytfitEs$  cCsdS(N((RoRCRD((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRuis(t__name__t __module__t__doc__RpR}Ru(((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRm8s  $tSelectPercentilecBs/eZdZedd„Zd„Zd„ZRS(s–Select features according to a percentile of the highest scores. Read more in the :ref:`User Guide `. Parameters ---------- score_func : callable Function taking two arrays X and y, and returning a pair of arrays (scores, pvalues) or a single array with scores. Default is f_classif (see below "See also"). The default function only works with classification tasks. percentile : int, optional, default=10 Percent of features to keep. Attributes ---------- scores_ : array-like, shape=(n_features,) Scores of features. pvalues_ : array-like, shape=(n_features,) p-values of feature scores, None if `score_func` returned only scores. Examples -------- >>> from sklearn.datasets import load_digits >>> from sklearn.feature_selection import SelectPercentile, chi2 >>> X, y = load_digits(return_X_y=True) >>> X.shape (1797, 64) >>> X_new = SelectPercentile(chi2, percentile=10).fit_transform(X, y) >>> X_new.shape (1797, 7) Notes ----- Ties between features with equal scores will be broken in an unspecified way. See also -------- f_classif: ANOVA F-value between label/feature for classification tasks. mutual_info_classif: Mutual information for a discrete target. chi2: Chi-squared stats of non-negative features for classification tasks. f_regression: F-value between label/feature for regression tasks. mutual_info_regression: Mutual information for a continuous target. SelectKBest: Select features based on the k highest scores. SelectFpr: Select features based on a false positive rate test. SelectFdr: Select features based on an estimated false discovery rate. SelectFwe: Select features based on family-wise error rate. GenericUnivariateSelect: Univariate feature selector with configurable mode. i cCs#tt|ƒj|ƒ||_dS(N(tsuperRRpt percentile(RoRnRƒ((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRp¦scCs9d|jkodkns5td|jƒ‚ndS(Niids'percentile should be >=0, <=100; got %r(RƒRR(RoRCRD((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRuªscCst|dƒ|jdkr;tjt|jƒdtjƒS|jdkritjt|jƒdtjƒSt|jƒ}tj|d|jƒ}||k}tj ||kƒd}t|ƒrt t|ƒ|jdƒ}|||j ƒ }t ||`. Parameters ---------- score_func : callable Function taking two arrays X and y, and returning a pair of arrays (scores, pvalues) or a single array with scores. Default is f_classif (see below "See also"). The default function only works with classification tasks. k : int or "all", optional, default=10 Number of top features to select. The "all" option bypasses selection, for use in a parameter search. Attributes ---------- scores_ : array-like, shape=(n_features,) Scores of features. pvalues_ : array-like, shape=(n_features,) p-values of feature scores, None if `score_func` returned only scores. Examples -------- >>> from sklearn.datasets import load_digits >>> from sklearn.feature_selection import SelectKBest, chi2 >>> X, y = load_digits(return_X_y=True) >>> X.shape (1797, 64) >>> X_new = SelectKBest(chi2, k=20).fit_transform(X, y) >>> X_new.shape (1797, 20) Notes ----- Ties between features with equal scores will be broken in an unspecified way. See also -------- f_classif: ANOVA F-value between label/feature for classification tasks. mutual_info_classif: Mutual information for a discrete target. chi2: Chi-squared stats of non-negative features for classification tasks. f_regression: F-value between label/feature for regression tasks. mutual_info_regression: Mutual information for a continuous target. SelectPercentile: Select features based on percentile of the highest scores. SelectFpr: Select features based on a false positive rate test. SelectFdr: Select features based on an estimated false discovery rate. SelectFwe: Select features based on family-wise error rate. GenericUnivariateSelect: Univariate feature selector with configurable mode. i cCs#tt|ƒj|ƒ||_dS(N(R‚RŽRpR4(RoRnR4((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRpúscCs\|jdkp2d|jko0|jdknsXtd|jd|jfƒ‚ndS(NtalliisPk should be >=0, <= n_features = %d; got %r. Use k='all' to return all features.(R4RRR(RoRCRD((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRuþs5cCs­t|dƒ|jdkr5tj|jjdtƒS|jdkr]tj|jjdtƒSt|jƒ}tj|jdtƒ}d|tj |ddƒ|j <|SdS(NRyRRiitkindt mergesort( R R4RR„RyRR…R†Rtargsort(RoRR‰((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRs !(R~RR€RERpRuR(((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRŽÃs5 t SelectFprcBs&eZdZedd„Zd„ZRS(s.Filter: Select the pvalues below alpha based on a FPR test. FPR test stands for False Positive Rate test. It controls the total amount of false detections. Read more in the :ref:`User Guide `. Parameters ---------- score_func : callable Function taking two arrays X and y, and returning a pair of arrays (scores, pvalues). Default is f_classif (see below "See also"). The default function only works with classification tasks. alpha : float, optional The highest p-value for features to be kept. Attributes ---------- scores_ : array-like, shape=(n_features,) Scores of features. pvalues_ : array-like, shape=(n_features,) p-values of feature scores. Examples -------- >>> from sklearn.datasets import load_breast_cancer >>> from sklearn.feature_selection import SelectFpr, chi2 >>> X, y = load_breast_cancer(return_X_y=True) >>> X.shape (569, 30) >>> X_new = SelectFpr(chi2, alpha=0.01).fit_transform(X, y) >>> X_new.shape (569, 16) See also -------- f_classif: ANOVA F-value between label/feature for classification tasks. chi2: Chi-squared stats of non-negative features for classification tasks. mutual_info_classif: f_regression: F-value between label/feature for regression tasks. mutual_info_regression: Mutual information between features and the target. SelectPercentile: Select features based on percentile of the highest scores. SelectKBest: Select features based on the k highest scores. SelectFdr: Select features based on an estimated false discovery rate. SelectFwe: Select features based on family-wise error rate. GenericUnivariateSelect: Univariate feature selector with configurable mode. gš™™™™™©?cCs#tt|ƒj|ƒ||_dS(N(R‚R“Rptalpha(RoRnR”((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRpIscCst|dƒ|j|jkS(NRy(R RzR”(Ro((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRMs (R~RR€RERpR(((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyR“s2t SelectFdrcBs&eZdZedd„Zd„ZRS(s»Filter: Select the p-values for an estimated false discovery rate This uses the Benjamini-Hochberg procedure. ``alpha`` is an upper bound on the expected false discovery rate. Read more in the :ref:`User Guide `. Parameters ---------- score_func : callable Function taking two arrays X and y, and returning a pair of arrays (scores, pvalues). Default is f_classif (see below "See also"). The default function only works with classification tasks. alpha : float, optional The highest uncorrected p-value for features to keep. Examples -------- >>> from sklearn.datasets import load_breast_cancer >>> from sklearn.feature_selection import SelectFdr, chi2 >>> X, y = load_breast_cancer(return_X_y=True) >>> X.shape (569, 30) >>> X_new = SelectFdr(chi2, alpha=0.01).fit_transform(X, y) >>> X_new.shape (569, 16) Attributes ---------- scores_ : array-like, shape=(n_features,) Scores of features. pvalues_ : array-like, shape=(n_features,) p-values of feature scores. References ---------- https://en.wikipedia.org/wiki/False_discovery_rate See also -------- f_classif: ANOVA F-value between label/feature for classification tasks. mutual_info_classif: Mutual information for a discrete target. chi2: Chi-squared stats of non-negative features for classification tasks. f_regression: F-value between label/feature for regression tasks. mutual_info_regression: Mutual information for a contnuous target. SelectPercentile: Select features based on percentile of the highest scores. SelectKBest: Select features based on the k highest scores. SelectFpr: Select features based on a false positive rate test. SelectFwe: Select features based on family-wise error rate. GenericUnivariateSelect: Univariate feature selector with configurable mode. gš™™™™™©?cCs#tt|ƒj|ƒ||_dS(N(R‚R•RpR”(RoRnR”((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRp‹scCs—t|dƒt|jƒ}tj|jƒ}||t|jƒ|tjd|dƒk}|jdkr„tj |jdt ƒS|j|j ƒkS(NRyiiR( R RRzRtsortRR”tarangeR#t zeros_likeR…tmax(Rot n_featurestsvtselected((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRs (R~RR€RERpR(((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyR•Ss6t SelectFwecBs&eZdZedd„Zd„ZRS(sqFilter: Select the p-values corresponding to Family-wise error rate Read more in the :ref:`User Guide `. Parameters ---------- score_func : callable Function taking two arrays X and y, and returning a pair of arrays (scores, pvalues). Default is f_classif (see below "See also"). The default function only works with classification tasks. alpha : float, optional The highest uncorrected p-value for features to keep. Examples -------- >>> from sklearn.datasets import load_breast_cancer >>> from sklearn.feature_selection import SelectFwe, chi2 >>> X, y = load_breast_cancer(return_X_y=True) >>> X.shape (569, 30) >>> X_new = SelectFwe(chi2, alpha=0.01).fit_transform(X, y) >>> X_new.shape (569, 15) Attributes ---------- scores_ : array-like, shape=(n_features,) Scores of features. pvalues_ : array-like, shape=(n_features,) p-values of feature scores. See also -------- f_classif: ANOVA F-value between label/feature for classification tasks. chi2: Chi-squared stats of non-negative features for classification tasks. f_regression: F-value between label/feature for regression tasks. SelectPercentile: Select features based on percentile of the highest scores. SelectKBest: Select features based on the k highest scores. SelectFpr: Select features based on a false positive rate test. SelectFdr: Select features based on an estimated false discovery rate. GenericUnivariateSelect: Univariate feature selector with configurable mode. gš™™™™™©?cCs#tt|ƒj|ƒ||_dS(N(R‚RRpR”(RoRnR”((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRpÊscCs*t|dƒ|j|jt|jƒkS(NRy(R RzR”R(Ro((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRÎs (R~RR€RERpR(((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyR›s-tGenericUnivariateSelectcBsdeZdZied6ed6ed6ed6ed6Ze ddd„Z d„Z d „Z d „Z RS( s•Univariate feature selector with configurable strategy. Read more in the :ref:`User Guide `. Parameters ---------- score_func : callable Function taking two arrays X and y, and returning a pair of arrays (scores, pvalues). For modes 'percentile' or 'kbest' it can return a single array scores. mode : {'percentile', 'k_best', 'fpr', 'fdr', 'fwe'} Feature selection mode. param : float or int depending on the feature selection mode Parameter of the corresponding mode. Attributes ---------- scores_ : array-like, shape=(n_features,) Scores of features. pvalues_ : array-like, shape=(n_features,) p-values of feature scores, None if `score_func` returned scores only. Examples -------- >>> from sklearn.datasets import load_breast_cancer >>> from sklearn.feature_selection import GenericUnivariateSelect, chi2 >>> X, y = load_breast_cancer(return_X_y=True) >>> X.shape (569, 30) >>> transformer = GenericUnivariateSelect(chi2, 'k_best', param=20) >>> X_new = transformer.fit_transform(X, y) >>> X_new.shape (569, 20) See also -------- f_classif: ANOVA F-value between label/feature for classification tasks. mutual_info_classif: Mutual information for a discrete target. chi2: Chi-squared stats of non-negative features for classification tasks. f_regression: F-value between label/feature for regression tasks. mutual_info_regression: Mutual information for a continuous target. SelectPercentile: Select features based on percentile of the highest scores. SelectKBest: Select features based on the k highest scores. SelectFpr: Select features based on a false positive rate test. SelectFdr: Select features based on an estimated false discovery rate. SelectFwe: Select features based on family-wise error rate. Rƒtk_besttfprtfdrtfwegñh㈵øä>cCs,tt|ƒj|ƒ||_||_dS(N(R‚RžRptmodetparam(RoRnR£R¤((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRps cCsT|j|jd|jƒ}|jƒ}|jdƒ|ji|j|d6|S(NRni(t_selection_modesR£Rnt_get_param_namestremovet set_paramsR¤(Rotselectortpossible_params((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyt_make_selectors   cCs]|j|jkrCtd|jjƒ|jt|jƒfƒ‚n|jƒj||ƒdS(Ns>The mode passed should be one of %s, %r, (type %s) was passed.(R£R¥RRtkeysRtR«Ru(RoRCRD((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRu$s cCs;t|dƒ|jƒ}|j|_|j|_|jƒS(NRy(R R«RzRyR(RoR©((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyR-s     (R~RR€RRŽR“R•RR¥RERpR«RuR(((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyRžÚs2   ('R€tnumpyRR$tscipyRRt scipy.sparseRtbaseRt preprocessingRtutilsRRRRR t utils.extmathR R tutils.validationR R RR>RERNR^RRlRmRRŽR“R•RRž(((sMlib/python2.7/site-packages/sklearn/feature_selection/univariate_selection.pyts.  (  P   ; Q8SR>H?