ó ‡ˆ\c@ sddlmZddlZddlmZddlmZddlm Z ddl m Z ddl m Z dd lmZdd lmZdd lmZdd lmZd „Zd„Zdd„Zdd„Zdededd„Zddedd„Zddedd„ZdS(iÿÿÿÿ(tdivisionN(tissparse(tdigammai(tmoves(tmutual_info_score(tNearestNeighbors(tscale(tcheck_random_state(t check_X_y(tcheck_classification_targetsc C s…|j}|jd ƒ}|jd ƒ}tj||fƒ}tddd|ƒ}|j|ƒ|jƒd}tj|dd…dfdƒ}|jdd ƒ|j|ƒ|j d |d t ƒ}tj g|D]}|j^qÒƒ} |j|ƒ|j d |d t ƒ}tj g|D]}|j^qƒ} t |ƒt |ƒtj t | dƒƒtj t | dƒƒ} td| ƒS(smCompute mutual information between two continuous variables. Parameters ---------- x, y : ndarray, shape (n_samples,) Samples of two continuous random variables, must have an identical shape. n_neighbors : int Number of nearest neighbors to search for each point, see [1]_. Returns ------- mi : float Estimated mutual information. If it turned out to be negative it is replace by 0. Notes ----- True mutual information can't be negative. If its estimate by a numerical method is negative, it means (providing the method is adequate) that the mutual information is close to 0 and replacing it by 0 is a reasonable strategy. References ---------- .. [1] A. Kraskov, H. Stogbauer and P. Grassberger, "Estimating mutual information". Phys. Rev. E 69, 2004. iÿÿÿÿitmetrict chebyshevt n_neighborsiNt algorithmtkd_treetradiustreturn_distance(iÿÿÿÿi(iÿÿÿÿi(tsizetreshapetnpthstackRtfitt kneighborst nextaftert set_paramstradius_neighborstFalsetarrayRtmeantmax( txtyR t n_samplestxytnnRtindtitnxtnytmi((sElib/python2.7/site-packages/sklearn/feature_selection/mutual_info_.pyt_compute_mi_ccs"  " % %DcC s|jd}|jd ƒ}tj|ƒ}tj|ƒ}tj|ƒ}tƒ}x¿tj|ƒD]®}||k} tj| ƒ} | dkrt|| dƒ} |jd| ƒ|j || ƒ|j ƒd} tj | dd…dfdƒ|| <| || tTruetanyt ValueErrorRRNRRtastypetfloattmaximumRtabstrandnR,RtzipRFR;R(RBRtdiscrete_featurestdiscrete_targetR RNt random_stateR t n_featurest discrete_masktcontinuous_masktrngtmeansRtdiscrete_featureR'((sElib/python2.7/site-packages/sklearn/feature_selection/mutual_info_.pyt _estimate_miÄs@4"   ":#<@cC st|||t|||ƒS(s Estimate mutual information for a continuous target variable. Mutual information (MI) [1]_ between two random variables is a non-negative value, which measures the dependency between the variables. It is equal to zero if and only if two random variables are independent, and higher values mean higher dependency. The function relies on nonparametric methods based on entropy estimation from k-nearest neighbors distances as described in [2]_ and [3]_. Both methods are based on the idea originally proposed in [4]_. It can be used for univariate features selection, read more in the :ref:`User Guide `. Parameters ---------- X : array_like or sparse matrix, shape (n_samples, n_features) Feature matrix. y : array_like, shape (n_samples,) Target vector. discrete_features : {'auto', bool, array_like}, default 'auto' If bool, then determines whether to consider all features discrete or continuous. If array, then it should be either a boolean mask with shape (n_features,) or array with indices of discrete features. If 'auto', it is assigned to False for dense `X` and to True for sparse `X`. n_neighbors : int, default 3 Number of neighbors to use for MI estimation for continuous variables, see [2]_ and [3]_. Higher values reduce variance of the estimation, but could introduce a bias. copy : bool, default True Whether to make a copy of the given data. If set to False, the initial data will be overwritten. random_state : int, RandomState instance or None, optional, default None The seed of the pseudo random number generator for adding small noise to continuous variables in order to remove repeated values. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Returns ------- mi : ndarray, shape (n_features,) Estimated mutual information between each feature and the target. Notes ----- 1. The term "discrete features" is used instead of naming them "categorical", because it describes the essence more accurately. For example, pixel intensities of an image are discrete features (but hardly categorical) and you will get better results if mark them as such. Also note, that treating a continuous variable as discrete and vice versa will usually give incorrect results, so be attentive about that. 2. True mutual information can't be negative. If its estimate turns out to be negative, it is replaced by zero. References ---------- .. [1] `Mutual Information `_ on Wikipedia. .. [2] A. Kraskov, H. Stogbauer and P. Grassberger, "Estimating mutual information". Phys. Rev. E 69, 2004. .. [3] B. C. Ross "Mutual Information between Discrete and Continuous Data Sets". PLoS ONE 9(2), 2014. .. [4] L. F. Kozachenko, N. N. Leonenko, "Sample Estimate of the Entropy of a Random Vector", Probl. Peredachi Inf., 23:2 (1987), 9-16 (ReR(RBRR\R RNR^((sElib/python2.7/site-packages/sklearn/feature_selection/mutual_info_.pytmutual_info_regression&sKcC s&t|ƒt|||t|||ƒS(s Estimate mutual information for a discrete target variable. Mutual information (MI) [1]_ between two random variables is a non-negative value, which measures the dependency between the variables. It is equal to zero if and only if two random variables are independent, and higher values mean higher dependency. The function relies on nonparametric methods based on entropy estimation from k-nearest neighbors distances as described in [2]_ and [3]_. Both methods are based on the idea originally proposed in [4]_. It can be used for univariate features selection, read more in the :ref:`User Guide `. Parameters ---------- X : array_like or sparse matrix, shape (n_samples, n_features) Feature matrix. y : array_like, shape (n_samples,) Target vector. discrete_features : {'auto', bool, array_like}, default 'auto' If bool, then determines whether to consider all features discrete or continuous. If array, then it should be either a boolean mask with shape (n_features,) or array with indices of discrete features. If 'auto', it is assigned to False for dense `X` and to True for sparse `X`. n_neighbors : int, default 3 Number of neighbors to use for MI estimation for continuous variables, see [2]_ and [3]_. Higher values reduce variance of the estimation, but could introduce a bias. copy : bool, default True Whether to make a copy of the given data. If set to False, the initial data will be overwritten. random_state : int, RandomState instance or None, optional, default None The seed of the pseudo random number generator for adding small noise to continuous variables in order to remove repeated values. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Returns ------- mi : ndarray, shape (n_features,) Estimated mutual information between each feature and the target. Notes ----- 1. The term "discrete features" is used instead of naming them "categorical", because it describes the essence more accurately. For example, pixel intensities of an image are discrete features (but hardly categorical) and you will get better results if mark them as such. Also note, that treating a continuous variable as discrete and vice versa will usually give incorrect results, so be attentive about that. 2. True mutual information can't be negative. If its estimate turns out to be negative, it is replaced by zero. References ---------- .. [1] `Mutual Information `_ on Wikipedia. .. [2] A. Kraskov, H. Stogbauer and P. Grassberger, "Estimating mutual information". Phys. Rev. E 69, 2004. .. [3] B. C. Ross "Mutual Information between Discrete and Continuous Data Sets". PLoS ONE 9(2), 2014. .. [4] L. F. Kozachenko, N. N. Leonenko, "Sample Estimate of the Entropy of a Random Vector:, Probl. Peredachi Inf., 23:2 (1987), 9-16 (R ReRS(RBRR\R RNR^((sElib/python2.7/site-packages/sklearn/feature_selection/mutual_info_.pytmutual_info_classifusK ( t __future__RtnumpyRt scipy.sparseRt scipy.specialRt externals.sixRtmetrics.cluster.supervisedRt neighborsRt preprocessingRtutilsRtutils.validationRtutils.multiclassR R(R8R;R<RFRRSReRfRg(((sElib/python2.7/site-packages/sklearn/feature_selection/mutual_info_.pyts(  = G  aN