ó ‡ˆ\c@sTdZddlZddlZddlZddljZddlmZddlm Z ddl m Z m Z ddl mZddlmZdd lmZdd lmZdd lmZdd l mZdd lmZejjZejjZddgZd„Zd„Zd„Zde e fd„ƒYZ de e fd„ƒYZ!dS(s)Transformers for missing value imputationiÿÿÿÿN(tsparse(tstatsi(t BaseEstimatortTransformerMixin(t check_array(t _get_median(tcheck_is_fitted(t FLOAT_DTYPES(t_object_dtype_isnan(t is_scalar_nan(tsixtMissingIndicatort SimpleImputercCsM|jjdkrIt|tjƒ rItdj|jt|ƒƒƒ‚ndS(Ntftitusn'X' and 'missing_values' types are expected to be both numerical. Got X.dtype={} and type(missing_values)={}.(R RR(tdtypetkindt isinstancetnumberstRealt ValueErrortformatttype(tXtmissing_values((s-lib/python2.7/site-packages/sklearn/impute.pyt_check_inputs_dtype!s cCstt|ƒr`|jjdkr+tj|ƒS|jjdkrStj|jdtƒSt|ƒSntj ||ƒSdS(s-Compute the boolean mask X == missing_values.R RRRN(RR( R RRtnptisnantzerostshapetboolRtequal(Rt value_to_mask((s-lib/python2.7/site-packages/sklearn/impute.pyt _get_mask*s   cCsÒ|jdkr`tjƒ$tjdtƒtj|ƒ}WdQX|dd}|dd}n d}d}|dkr‹|dkr‹tjS||kr›|S||kr«|S||krÎ||krÇ|S|SndS(sCompute the most frequent value in a 1d array extended with [extra_value] * n_repeat, where extra_value is assumed to be not part of the array.itignoreNi( tsizetwarningstcatch_warningst simplefiltertRuntimeWarningRtmodeRtnan(tarrayt extra_valuetn_repeatR)tmost_frequent_valuetmost_frequent_count((s-lib/python2.7/site-packages/sklearn/impute.pyt_most_frequent;s$     cBsYeZdZejdd ded„Zd„Zd d„Z d„Z d„Z d„Z RS( s_ Imputation transformer for completing missing values. Read more in the :ref:`User Guide `. Parameters ---------- missing_values : number, string, np.nan (default) or None The placeholder for the missing values. All occurrences of `missing_values` will be imputed. strategy : string, optional (default="mean") The imputation strategy. - If "mean", then replace missing values using the mean along each column. Can only be used with numeric data. - If "median", then replace missing values using the median along each column. Can only be used with numeric data. - If "most_frequent", then replace missing using the most frequent value along each column. Can be used with strings or numeric data. - If "constant", then replace missing values with fill_value. Can be used with strings or numeric data. .. versionadded:: 0.20 strategy="constant" for fixed value imputation. fill_value : string or numerical value, optional (default=None) When strategy == "constant", fill_value is used to replace all occurrences of missing_values. If left to the default, fill_value will be 0 when imputing numerical data and "missing_value" for strings or object data types. verbose : integer, optional (default=0) Controls the verbosity of the imputer. copy : boolean, optional (default=True) If True, a copy of X will be created. If False, imputation will be done in-place whenever possible. Note that, in the following cases, a new copy will always be made, even if `copy=False`: - If X is not an array of floating values; - If X is encoded as a CSR matrix. Attributes ---------- statistics_ : array of shape (n_features,) The imputation fill value for each feature. Examples -------- >>> import numpy as np >>> from sklearn.impute import SimpleImputer >>> imp_mean = SimpleImputer(missing_values=np.nan, strategy='mean') >>> imp_mean.fit([[7, 2, 3], [4, np.nan, 6], [10, 5, 9]]) ... # doctest: +NORMALIZE_WHITESPACE SimpleImputer(copy=True, fill_value=None, missing_values=nan, strategy='mean', verbose=0) >>> X = [[np.nan, 2, 3], [4, np.nan, 6], [10, np.nan, 9]] >>> print(imp_mean.transform(X)) ... # doctest: +NORMALIZE_WHITESPACE [[ 7. 2. 3. ] [ 4. 3.5 6. ] [10. 3.5 9. ]] Notes ----- Columns which only contained missing values at `fit` are discarded upon `transform` if strategy is not "constant". tmeanicCs1||_||_||_||_||_dS(N(Rtstrategyt fill_valuetverbosetcopy(tselfRR2R3R4R5((s-lib/python2.7/site-packages/sklearn/impute.pyt__init__£s     c Cs9ddddg}|j|kr?tdj||jƒƒ‚n|jdkrWd}nt}t|jƒsut}nd}y+t|ddd |d |d |j ƒ}WnOtk r÷}d t |ƒkrîtd j|j|j j ƒƒ‚qø|‚nXt ||jƒ|j j dkr5tdj|j ƒƒ‚n|S(NR1tmediant most_frequenttconstants4Can only use these strategies: {0} got strategy={1}s allow-nant accept_sparsetcscRtforce_all_finiteR5scould not convertsFCannot use {0} strategy with non-numeric data. Received datatype :{1}.RRR tOsûSimpleImputer does not support data with dtype {0}. Please provide either a numeric array (with a floating point or integer dtype) or categorical data represented either as an array with integer dtype or an array of string values with an object dtype.(R9R:(RRR R>(R2RRtNoneRR RtTrueRR5tstrRRR(R6Rtallowed_strategiesRR=tve((s-lib/python2.7/site-packages/sklearn/impute.pyt_validate_input«s0      cCs |j|ƒ}|jd krB|jjd kr9d}qKd}n |j}|jdkr—|jjd kr—t|tjƒ r—t dj |ƒƒ‚nt j |ƒrè|j dkrÄt dƒ‚q |j||j|j |ƒ|_n!|j||j|j |ƒ|_|S( sIFit the imputer on X. Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) Input data, where ``n_samples`` is the number of samples and ``n_features`` is the number of features. Returns ------- self : SimpleImputer RRR it missing_valueR:sT'fill_value'={0} is invalid. Expected a numerical value when imputing numerical datasdImputation not possible when missing_values == 0 and input is sparse. Provide a dense array instead.N(RRR (RRR (RDR3R?RRR2RRRRRRtissparseRt _sparse_fitt statistics_t _dense_fit(R6RtyR3((s-lib/python2.7/site-packages/sklearn/impute.pytfitÒs.        cCst|j|ƒ}|jdtj|jƒ}tj|jdƒ}|dkra|j|ƒn*x't|jdƒD]}|j|j||j|d!} ||j||j|d!} | | } t| dƒ} | | } | j ƒ} ||| } |dkr@| j | }|dkr)tj n | j ƒ|||RR:iN(R"tmat masked_arrayRR1tgetdataR*tgetmaskR8t getmaskarrayt transposeRRRORtobjectt enumeratetzipt logical_nottastypeRR0tfull(R6RR2RR3R\tmasked_Xt mean_maskedR1t median_maskedR8R9Rtrowtrow_mask((s-lib/python2.7/site-packages/sklearn/impute.pyRI-s2     *  c Cs)t|dƒ|j|ƒ}|j}|jd|jdkritd|jd|jjdfƒ‚n|jdkr|}n–t|tjƒ}tj |ƒ}||}tj |ƒ}|j ƒrtj |jdƒ|}|j rþtjd|ƒn|dd…|f}ntj|ƒrÀ|jdkrDtdƒ‚q%t|j|jƒ}tjtj t|jƒdd tjƒtj|jƒƒ|} || j|jd tƒ|j|`. Parameters ---------- missing_values : number, string, np.nan (default) or None The placeholder for the missing values. All occurrences of `missing_values` will be indicated (True in the output array), the other values will be marked as False. features : str, optional Whether the imputer mask should represent all or a subset of features. - If "missing-only" (default), the imputer mask will only represent features containing missing values during fit time. - If "all", the imputer mask will represent all features. sparse : boolean or "auto", optional Whether the imputer mask format should be sparse or dense. - If "auto" (default), the imputer mask will be of same type as input. - If True, the imputer mask will be a sparse matrix. - If False, the imputer mask will be a numpy array. error_on_new : boolean, optional If True (default), transform will raise an error when there are features with missing values in transform that have no missing values in fit. This is applicable only when ``features="missing-only"``. Attributes ---------- features_ : ndarray, shape (n_missing_features,) or (n_features,) The features indices which will be returned when calling ``transform``. They are computed during ``fit``. For ``features='all'``, it is to ``range(n_features)``. Examples -------- >>> import numpy as np >>> from sklearn.impute import MissingIndicator >>> X1 = np.array([[np.nan, 1, 3], ... [4, 0, np.nan], ... [8, 1, 0]]) >>> X2 = np.array([[5, 1, np.nan], ... [np.nan, 2, 3], ... [2, 4, 0]]) >>> indicator = MissingIndicator() >>> indicator.fit(X1) MissingIndicator(error_on_new=True, features='missing-only', missing_values=nan, sparse='auto') >>> X2_tr = indicator.transform(X2) >>> X2_tr array([[False, True], [ True, False], [False, False]]) s missing-onlytautocCs(||_||_||_||_dS(N(RtfeaturesRt error_on_new(R6RR†RR‡((s-lib/python2.7/site-packages/sklearn/impute.pyR7Þs   cCs‘tj|ƒr|jdkrt|j|jƒ}|jdkrKtjntj}|||jj ƒ|j j ƒfd|j dt ƒ}|j ƒ}|j ƒ|jdkrÊtjtj|j ƒƒntj|jƒ}|jtkrú|jƒ}q‡|jdkr‡|jƒ}q‡nltj|ƒr9|jƒ}nt||jƒ}tj|jddƒƒ}|jtkr‡tj|ƒ}n||fS(sµCompute the imputer mask and the indices of the features containing missing values. Parameters ---------- X : {ndarray or sparse matrix}, shape (n_samples, n_features) The input data with missing values. Note that ``X`` has been checked in ``fit`` and ``transform`` before to call this function. Returns ------- imputer_mask : {ndarray or sparse matrix}, shape (n_samples, n_features) or (n_samples, n_features_with_missing) The imputer mask of the original data. features_with_missing : ndarray, shape (n_features_with_missing) The features containing missing values. itcsrRRR<R](RRFRR"RLRt csr_matrixt csc_matrixtindicesR5RNRRteliminate_zerosRRoRMtuniqueRvttoarrayttocscRRR@(R6RR\tsparse_constructort imputer_masktmissing_values_masktfeatures_with_missing((s-lib/python2.7/site-packages/sklearn/impute.pyt_get_missing_features_infoås, !  *cCs}t|jƒst}nd}t|dd ddd|ƒ}t||jƒ|jjd krytd j |jƒƒ‚n|S(Ns allow-nanR;R<RˆRR=RRR R>sþMissingIndicator does not support data with dtype {0}. Please provide either a numeric array (with a floating point or integer dtype) or categorical data represented either as an array with integer dtype or an array of string values with an object dtype.(R<Rˆ(RRR R>( R RR@RR?RRRRR(R6RR=((s-lib/python2.7/site-packages/sklearn/impute.pyRDs   cCsÕ|j|ƒ}|jd|_|jdkrItdj|jƒƒ‚nt|jtj ƒrm|jdkp|t|jt ƒsštdj|jƒƒ‚n|jdkr¼|j |ƒdnt j |jƒ|_|S(s`Fit the transformer on X. Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) Input data, where ``n_samples`` is the number of samples and ``n_features`` is the number of features. Returns ------- self : object Returns self. is missing-onlytallsD'features' has to be either 'missing-only' or 'all'. Got {} instead.R…s9'sparse' has to be a boolean or 'auto'. Got {!r} instead.(s missing-onlyR•(RDRt _n_featuresR†RRRRR t string_typesRR”RRqt features_(R6RRJ((s-lib/python2.7/site-packages/sklearn/impute.pyRK-s ! "cCsôt|dƒ|j|ƒ}|jd|jkrAtdƒ‚n|j|ƒ\}}|jdkrðtj||j ƒ}|j rª|j dkrªtdj |ƒƒ‚n|j j dkrð|j j |jkrð|dd…|j f}qðn|S(s—Generate missing values indicator for X. Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) The input data to complete. Returns ------- Xt : {ndarray or sparse matrix}, shape (n_samples, n_features) The missing indicator for input data. The data type of ``Xt`` will be boolean. R˜is9X has a different number of features than during fitting.s missing-onlyisSThe features {} have missing values in transform but have no missing values in fit.N( RRDRR–RR”R†Rt setdiff1dR˜R‡R$R(R6RR‘R†tfeatures_diff_fit_trans((s-lib/python2.7/site-packages/sklearn/impute.pyRMs  cCs|j||ƒj|ƒS(s—Generate missing values indicator for X. Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) The input data to complete. Returns ------- Xt : {ndarray or sparse matrix}, shape (n_samples, n_features) The missing indicator for input data. The data type of ``Xt`` will be boolean. (RKR(R6RRJ((s-lib/python2.7/site-packages/sklearn/impute.pyt fit_transformrsN( R‚RƒR„RR*R@R7R”RDR?RKRR›(((s-lib/python2.7/site-packages/sklearn/impute.pyR œs@  7  %("R„R%RtnumpyRtnumpy.maR^tscipyRRtbaseRRtutilsRtutils.sparsefuncsRtutils.validationRRt utils.fixesRR t externalsR tmovesRftmapt__all__RR"R0R R (((s-lib/python2.7/site-packages/sklearn/impute.pyts0        "ÿ@