ó ‡ˆ\c@ sldZddlmZddlmZddlZddlZddlm Z ddl m Z m Z ddl mZmZdd lmZdd lmZmZmZdd lmZdd lmZdd lmZddlmZmZddgZdZ dee fd„ƒYZ!d„Z"d„Z#d„Z$d„Z%d„Z&d„Z'd„Z(d„Z)d„Z*dS(s¦ The :mod:`sklearn.compose._column_transformer` module implements utilities to work with heterogeneous data and to apply different transformers to different columns. iÿÿÿÿ(tdivision(tchainN(tsparsei(tclonetTransformerMixin(tParalleltdelayed(tsix(t_fit_transform_onet_transform_onet_name_estimators(tFunctionTransformer(tBunch(t_BaseComposition(t check_arraytcheck_is_fittedtColumnTransformertmake_column_transformers„1D data passed to a transformer that expects 2D data. Try to specify the column selection as a list of one item instead of a scalar.cB sãeZdZddddd„Zed„ƒZejd„ƒZed„Z d„Z e e d„Z d „Z d „Zd „Zed „ƒZd „Zd„Zd„Ze d„Zdd„Zdd„Zd„Zd„ZRS(szApplies transformers to columns of an array or pandas DataFrame. EXPERIMENTAL: some behaviors may change between releases without deprecation. This estimator allows different columns or column subsets of the input to be transformed separately and the features generated by each transformer will be concatenated to form a single feature space. This is useful for heterogeneous or columnar data, to combine several feature extraction mechanisms or transformations into a single transformer. Read more in the :ref:`User Guide `. .. versionadded:: 0.20 Parameters ---------- transformers : list of tuples List of (name, transformer, column(s)) tuples specifying the transformer objects to be applied to subsets of the data. name : string Like in Pipeline and FeatureUnion, this allows the transformer and its parameters to be set using ``set_params`` and searched in grid search. transformer : estimator or {'passthrough', 'drop'} Estimator must support `fit` and `transform`. Special-cased strings 'drop' and 'passthrough' are accepted as well, to indicate to drop the columns or to pass them through untransformed, respectively. column(s) : string or int, array-like of string or int, slice, boolean mask array or callable Indexes the data on its second axis. Integers are interpreted as positional columns, while strings can reference DataFrame columns by name. A scalar string or int should be used where ``transformer`` expects X to be a 1d array-like (vector), otherwise a 2d array will be passed to the transformer. A callable is passed the input data `X` and can return any of the above. remainder : {'drop', 'passthrough'} or estimator, default 'drop' By default, only the specified columns in `transformers` are transformed and combined in the output, and the non-specified columns are dropped. (default of ``'drop'``). By specifying ``remainder='passthrough'``, all remaining columns that were not specified in `transformers` will be automatically passed through. This subset of columns is concatenated with the output of the transformers. By setting ``remainder`` to be an estimator, the remaining non-specified columns will use the ``remainder`` estimator. The estimator must support `fit` and `transform`. sparse_threshold : float, default = 0.3 If the output of the different transfromers contains sparse matrices, these will be stacked as a sparse matrix if the overall density is lower than this value. Use ``sparse_threshold=0`` to always return dense. When the transformed output consists of all dense data, the stacked result will be dense, and this keyword will be ignored. n_jobs : int or None, optional (default=None) Number of jobs to run in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. transformer_weights : dict, optional Multiplicative weights for features per transformer. The output of the transformer is multiplied by these weights. Keys are transformer names, values the weights. Attributes ---------- transformers_ : list The collection of fitted transformers as tuples of (name, fitted_transformer, column). `fitted_transformer` can be an estimator, 'drop', or 'passthrough'. In case there were no columns selected, this will be the unfitted transformer. If there are remaining columns, the final element is a tuple of the form: ('remainder', transformer, remaining_columns) corresponding to the ``remainder`` parameter. If there are remaining columns, then ``len(transformers_)==len(transformers)+1``, otherwise ``len(transformers_)==len(transformers)``. named_transformers_ : Bunch object, a dictionary with attribute access Read-only attribute to access any transformer by given name. Keys are transformer names and values are the fitted transformer objects. sparse_output_ : boolean Boolean flag indicating wether the output of ``transform`` is a sparse matrix or a dense numpy array, which depends on the output of the individual transformers and the `sparse_threshold` keyword. Notes ----- The order of the columns in the transformed feature matrix follows the order of how the columns are specified in the `transformers` list. Columns of the original feature matrix that are not specified are dropped from the resulting transformed feature matrix, unless specified in the `passthrough` keyword. Those columns specified with `passthrough` are added at the right to the output of the transformers. See also -------- sklearn.compose.make_column_transformer : convenience function for combining the outputs of multiple transformer objects applied to column subsets of the original feature space. Examples -------- >>> from sklearn.compose import ColumnTransformer >>> from sklearn.preprocessing import Normalizer >>> ct = ColumnTransformer( ... [("norm1", Normalizer(norm='l1'), [0, 1]), ... ("norm2", Normalizer(norm='l1'), slice(2, 4))]) >>> X = np.array([[0., 1., 2., 2.], ... [1., 1., 0., 1.]]) >>> # Normalizer scales each row of X to unit norm. A separate scaling >>> # is applied for the two first and two last elements of each >>> # row independently. >>> ct.fit_transform(X) # doctest: +NORMALIZE_WHITESPACE array([[0. , 1. , 0.5, 0.5], [0.5, 0.5, 0. , 1. ]]) tdropg333333Ó?cC s1||_||_||_||_||_dS(N(t transformerst remaindertsparse_thresholdtn_jobsttransformer_weights(tselfRRRRR((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt__init__£s     cC s)g|jD]\}}}||f^q S(sü Internal list of transformer only containing the name and transformers, dropping the columns. This is for the implementation of get_params via BaseComposition._get_params which expects lists of tuples of len 2. (R(Rtnamettranst_((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt _transformers«scC sJgt||jƒD]*\\}}\}}}|||f^q|_dS(N(tzipR(RtvalueRRRtcol((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyRµscC s|jdd|ƒS(soGet parameters for this estimator. Parameters ---------- deep : boolean, optional If True, will return the parameters for this estimator and contained subobjects that are estimators. Returns ------- params : mapping of string to any Parameter names mapped to their values. Rtdeep(t _get_params(RR!((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt get_params»scK s|jd||S(sŸSet the parameters of this estimator. Valid parameter keys can be listed with ``get_params()``. Returns ------- self R(t _set_params(Rtkwargs((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt set_paramsËs c c s|r|j}nkgt|j|jƒD]$\\}}}}|||f^q(}|jddk r}t||jgƒ}n|jp‰ij}x…|D]}\}}}|rü|dkrÕt dt dt dt ƒ}qü|dkrçq–qüt |ƒrüq–qün|||||ƒfVq–WdS(s  Generate (name, trans, column, weight) tuples. If fitted=True, use the fitted transformers, else use the user specified transformers updated with converted column names and potentially appended with transformer for remainder. it passthroughtvalidatet accept_sparset check_inverseRN( t transformers_RRt_columnst _remaindertNoneRRtgetR tFalsetTruet_is_empty_column_selection( Rtfittedtreplace_stringsRRRRtcolumnt get_weight((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt_iter×s$  =      cC s§|js dSt|jŒ\}}}|j|ƒxn|D]f}|dkrQq9nt|dƒplt|dƒ s€t|dƒ r9td|t|ƒfƒ‚q9q9WdS(NRR'tfitt fit_transformt transformsxAll estimators should implement fit and transform, or can be 'drop' or 'passthrough' specifiers. '%s' (type %s) doesn't.(RR'(RRt_validate_namesthasattrt TypeErrorttype(RtnamesRRtt((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt_validate_transformersüs    cC sXg}xB|jD]7\}}}t|ƒr:||ƒ}n|j|ƒqW||_dS(s: Converts callable column specifications. N(RtcallabletappendR,(RtXtcolumnsRR5((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt_validate_column_callabless  cC sæt|jdƒs$t|jdƒo3t|jdƒ}|jd krb| rbtd|jƒ‚n|jd}g}x'|jD]}|jt||ƒƒqWttt t |ƒƒt |ƒƒƒpÊd }d|j|f|_ d S( sm Validates ``remainder`` and defines ``_remainder`` targeting the remaining columns. R8R9R:RR'seThe remainder keyword needs to be one of 'drop', 'passthrough', or estimator. '%s' was passed insteadiRN(RR'( R<Rt ValueErrortshapeR,textendt_get_column_indicestsortedtlisttsettrangeR.R-(RRDtis_transformert n_columnstcolsREt remaining_idx((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt_validate_remainders .cC s5ttg|jD]\}}}||f^qƒS(sÐAccess the fitted transformer by name. Read-only attribute to access any transformer by given name. Keys are transformer names and values are the fitted transformer objects. (R tdictR+(RRRR((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pytnamed_transformers_2s cC sÒt|dƒg}x¸|jdtƒD]¤\}}}}|dkrJq&nR|dkretdƒ‚n7t|dƒsœtdt|ƒt|ƒjfƒ‚n|j g|j ƒD]}|d|^q¯ƒq&W|S( s³Get feature names from all transformers. Returns ------- feature_names : list of strings Names of the features produced by transform. R+R3RR'sNget_feature_names is not yet supported when using a 'passthrough' transformer.tget_feature_namess<Transformer %s (type %s) does not provide get_feature_names.t__( RR7R1tNotImplementedErrorR<tAttributeErrortstrR>t__name__RIRV(Rt feature_namesRRRtf((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyRV?s %   " )c C sÀt|ƒ}g}x‹|jƒD]}\}}}}|dkrFd}n@|dkret|ƒd}n!t|ƒrz|}n t|ƒ}|j|||fƒqWt|ƒ s³t‚||_dS(NRR'(titerR7tnextR2RCRLtAssertionErrorR+( RRtfitted_transformersR+RtoldR5RR((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt_update_fitted_transformersXs         cC s‹g|jdtdtƒD]\}}}}|^q}xMt||ƒD]<\}}t|ddƒdksGtdj|ƒƒ‚qGqGWdS(s Ensure that the output of each transformer is 2D. Otherwise hstack can raise an error or produce incorrect results. R3R4tndimiis\The output of the '{0}' transformer should be 2D (scipy matrix, array, or pandas DataFrame).N(R7R1RtgetattrRGtformat(RtresultRRR?tXs((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt_validate_outputos%c sƒyEtd|jƒ‡‡‡‡fd†|jdˆdtƒDƒƒSWn7tk r~}dt|ƒkrxttƒ‚q‚nXdS(sî Private function to fit and/or transform on demand. Return value (transformers and/or transformed X data) depends on the passed function. ``fitted=True`` ensures the fitted transformers are used. Rc3 sQ|]G\}}}}tˆƒˆs0t|ƒn|tˆ|ƒˆ|ƒVqdS(N(RRt _get_column(t.0RRR5tweight(RDR3tfuncty(sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pys †sR3R4s'Expected 2D array, got 1D array insteadN(RRR7R1RGRZt_ERR_MSG_1DCOLUMN(RRDRnRmR3te((RDR3RmRnsBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt_fit_transform|s cC s|j|d|ƒ|S(s¸Fit all transformers using X. Parameters ---------- X : array-like or DataFrame of shape [n_samples, n_features] Input data, of which specified subsets are used to fit the transformers. y : array-like, shape (n_samples, ...), optional Targets for supervised learning. Returns ------- self : ColumnTransformer This estimator Rn(R9(RRDRn((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyR8sc C st|ƒ}|jƒ|j|ƒ|j|ƒ|j||tƒ}|sr|jgƒtj|j ddfƒSt |Œ\}}t d„|Dƒƒråt d„|Dƒƒ}t d„|Dƒƒ}||}||j k|_n t|_|j|ƒ|j|ƒ|jt|ƒƒS(sçFit all transformers, transform the data and concatenate results. Parameters ---------- X : array-like or DataFrame of shape [n_samples, n_features] Input data, of which specified subsets are used to fit the transformers. y : array-like, shape (n_samples, ...), optional Targets for supervised learning. Returns ------- X_t : array-like or sparse matrix, shape (n_samples, sum_n_components) hstack of results of transformers. sum_n_components is the sum of n_components (output dimension) over transformers. If any result is a sparse matrix, everything will be converted to sparse matrices. ics s|]}tj|ƒVqdS(N(Rtissparse(RkRD((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pys Ëscs s0|]&}tj|ƒr!|jn|jVqdS(N(RRrtnnztsize(RkRD((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pys Ìscs s?|]5}tj|ƒr0|jd|jdn|jVqdS(iiN(RRrRHRt(RkRD((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pys Ís(t_check_XRARFRSRqRRctnptzerosRHRtanytsumRtsparse_output_R0Rit_hstackRL( RRDRnRgRhRRsttotaltdensity((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyR9§s&           cC stt|dƒt|ƒ}|j|dtdtƒ}|j|ƒ|satj|j ddfƒS|j t |ƒƒS(sPTransform X separately by each transformer, concatenate results. Parameters ---------- X : array-like or DataFrame of shape [n_samples, n_features] The data to be transformed by subset. Returns ------- X_t : array-like or sparse matrix, shape (n_samples, sum_n_components) hstack of results of transformers. sum_n_components is the sum of n_components (output dimension) over transformers. If any result is a sparse matrix, everything will be converted to sparse matrices. R+R3iN( RRuRqR.R R1RiRvRwRHR{RL(RRDRh((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyR:Ùs   cC s°|jrky/g|D]}t|dtdtƒ^q}Wntk rWtdƒ‚nXtj|ƒjƒSg|D]'}tj|ƒr“|j ƒn|^qr}t j|ƒSdS(sStacks Xs horizontally. This allows subclasses to control the stacking behavior, while reusing everything else from ColumnTransformer. Parameters ---------- Xs : List of numpy arrays, sparse arrays, or DataFrames R)tforce_all_finitesQFor a sparse output, all columns should be a numeric or convertible to a numeric.N( RzRR1R0RGRthstackttocsrRrttoarrayRv(RRhRDt converted_XsR]((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyR{ös , 4N(R[t __module__t__doc__R.RtpropertyRtsetterR1R#R&R0R7RARFRSRURVRcRiRqR8R9R:R{(((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyR#s(~   %       2 cC s;t|dƒstj|ƒr"|St|dddtjƒS(s@Use check_array only on lists and other non-array-likes / sparset __array__R~s allow-nantdtype(R<RRrRRvtobject(RD((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyRusc sÇt|ˆƒrtSt|tƒr\t|jˆtdƒfƒo[t|jˆtdƒfƒSt|tƒr…t‡fd†|DƒƒSt |dƒrÈt kr°|j j dkS|j j dkSnt S( s¡ Check that scalar, list or slice is of a certain type. This is only used in _get_column and _get_column_indices to check if the `key` (column specification) is fully integer or fully string-like. Parameters ---------- key : scalar, list, slice, array-like The column specification to check superclass : int or six.string_types The type for which to check the `key` c3 s|]}t|ˆƒVqdS(N(t isinstance(Rktx(t superclass(sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pys /sRˆtitOtUtSN(RŽRR(RŠR1tslicetstartR>R.tstopRLtallR<tintRˆtkindR0(tkeyRŒ((RŒsBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt_check_key_types cC st|tƒrt}not|tjƒr3t}nTt|dƒr{tj|j tj ƒr{t}t|dƒr‡t}q‡n t dƒ‚|rÂt|dƒr³|j dd…|fSt dƒ‚n:t|dƒrè|j dd…|fS|dd…|fSdS(s_ Get feature column(s) from input data X. Supported input types (X): numpy arrays, sparse arrays and DataFrames Supported key types (key): - scalar: output is 1D - lists, slices, boolean masks: output is 2D - callable that returns any of the above Supported key data types: - integer or boolean mask (positional): - supported for arrays, sparse matrices and dataframes - string (key-based): - only supported for dataframes - So no keys other than strings are allowed (while in principle you can use any hashable object as key). Rˆtlocs~No valid specification of the columns. Only a scalar, list or slice of all integers or all strings, or boolean mask is allowedNsLSpecifying the columns using strings is only supported for pandas DataFramestiloc(R˜R•R0Rt string_typesR1R<Rvt issubdtypeRˆtbool_RGR™Rš(RDR—t column_names((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyRj9s   '  c C s‘|jd}t|tƒsCt|dƒritj|jtjƒritj|ƒ|}tj |ƒj ƒSt|t j ƒryt |jƒ}Wntk r­tdƒ‚nXt|t j ƒrÌ|g}n•t|tƒrU|j|j}}|dk r |j|ƒ}n|dk r.|j|ƒd}n |d}t t|ƒt||ƒƒSt |ƒ}g|D]}|j|ƒ^qhStdƒ‚dS(sƒ Get feature column indices for input data X and key. For accepted values of `key`, see the docstring of _get_column iRˆsLSpecifying the columns using strings is only supported for pandas DataFramess~No valid specification of the columns. Only a scalar, list or slice of all integers or all strings, or boolean mask is allowedN(RHR˜R•R<RvRœRˆRtaranget atleast_1dttolistRR›RLRERYRGRŠR‘R’R“R.tindexRN( RDR—RPtidxt all_columnsRER’R“R ((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyRJns. '       cC sYt|dƒr2tj|jtjƒr2|jƒ St|dƒrQt|ƒdkStSdS(sd Return True if the column selection is empty (empty list or all-False boolean array). Rˆt__len__iN(R<RvRœRˆRRxtlenR0(R5((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyR2˜s ' cC sv|s tSxe|D]]}t|tjƒr;|dkr;qnt|dƒpVt|dƒ sjt|dƒ rtSqWtS(sŠChecks if given transformers are valid. This is a helper function to support the deprecated tuple order. XXX Remove in v0.22 RR'R8R9R:(RR'(R1RŠRR›R<R0(RR@((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyRA¦s cC s3t|Œ\}}t|ƒ r/t|ƒr/tStS(s,Checks if the input follows the deprecated tuple order. Returns ------- Returns true if (transformer, columns) is not a valid assumption for the input, but (columns, transformer) is valid. The latter is deprecated and its support will stop in v0.22. XXX Remove in v0.22 (RRAR1R0(ttuplesRRE((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt_is_deprecated_tuple_order¹s   cC sxd}t|Œ\}}t|ƒrD||}}tj|tƒntt|ƒŒ\}}tt|||ƒƒ}|S(s; Construct (name, trans, column) tuples from list sø`make_column_transformer` now expects (transformer, columns) as input tuples instead of (columns, transformer). This has been introduced in v0.20.1. `make_column_transformer` will stop accepting the deprecated (columns, transformer) order in v0.22.(RR¨twarningstwarntDeprecationWarningR RL(t estimatorstmessageRRER?Rttransformer_list((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyt_get_transformer_listÌs  cO sŒ|jddƒ}|jddƒ}|jddƒ}|rdtdjt|jƒƒdƒƒ‚nt|ƒ}t|d|d|d|ƒS( sX Construct a ColumnTransformer from the given transformers. This is a shorthand for the ColumnTransformer constructor; it does not require, and does not permit, naming the transformers. Instead, they will be given names automatically based on their types. It also does not allow weighting with ``transformer_weights``. Parameters ---------- *transformers : tuples of transformers and column selections remainder : {'drop', 'passthrough'} or estimator, default 'drop' By default, only the specified columns in `transformers` are transformed and combined in the output, and the non-specified columns are dropped. (default of ``'drop'``). By specifying ``remainder='passthrough'``, all remaining columns that were not specified in `transformers` will be automatically passed through. This subset of columns is concatenated with the output of the transformers. By setting ``remainder`` to be an estimator, the remaining non-specified columns will use the ``remainder`` estimator. The estimator must support `fit` and `transform`. sparse_threshold : float, default = 0.3 If the transformed output consists of a mix of sparse and dense data, it will be stacked as a sparse matrix if the density is lower than this value. Use ``sparse_threshold=0`` to always return dense. When the transformed output consists of all sparse or all dense data, the stacked result will be sparse or dense, respectively, and this keyword will be ignored. n_jobs : int or None, optional (default=None) Number of jobs to run in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Returns ------- ct : ColumnTransformer See also -------- sklearn.compose.ColumnTransformer : Class that allows combining the outputs of multiple transformer objects used on column subsets of the data into a single feature space. Examples -------- >>> from sklearn.preprocessing import StandardScaler, OneHotEncoder >>> from sklearn.compose import make_column_transformer >>> make_column_transformer( ... (StandardScaler(), ['numerical_column']), ... (OneHotEncoder(), ['categorical_column'])) ... # doctest: +NORMALIZE_WHITESPACE +ELLIPSIS ColumnTransformer(n_jobs=None, remainder='drop', sparse_threshold=0.3, transformer_weights=None, transformers=[('standardscaler', StandardScaler(...), ['numerical_column']), ('onehotencoder', OneHotEncoder(...), ['categorical_column'])]) RRRRg333333Ó?sUnknown keyword arguments: "{}"iN(tpopR.R=RfRLtkeysR¯R(RR%RRRR®((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyRäsD  (+R„t __future__Rt itertoolsRtnumpyRvR©tscipyRtbaseRRt utils._joblibRRt externalsRtpipelineRR R t preprocessingR tutilsR tutils.metaestimatorsR tutils.validationRRt__all__RoRRuR˜RjRJR2RAR¨R¯R(((sBlib/python2.7/site-packages/sklearn/compose/_column_transformer.pyts4   ÿñ   5 *