B T\y$@sddlmZmZmZddlZddlZddlm Z m Z m Z m Z ddl mZddlmZmZmZmZddddddejfd d Zdd d ZdddZdS))absolute_importdivisionprint_functionN)Series DataFramemap_partitionsapply_concat_apply)methods)is_categorical_dtype is_scalarhas_known_categoriesPANDAS_VERSION_Fc sXtdkrd|i}n|tjkr,tdtni}ttjtjfrbtj f||||||d|Sd} d} ttrt st | t st | n|ttr|dkrԈj dkrt | jjd gd j}ntfd d |Dst | tfd d |Dst | tj jf||||||d|} ttj f||||||| d|S)a Convert categorical variable into dummy/indicator variables. Data must have category dtype to infer result's ``columns``. Parameters ---------- data : Series, or DataFrame For Series, the dtype must be categorical. For DataFrame, at least one column must be categorical. prefix : string, list of strings, or dict of strings, default None String to append DataFrame column names. Pass a list with length equal to the number of columns when calling get_dummies on a DataFrame. Alternatively, `prefix` can be a dictionary mapping column names to prefixes. prefix_sep : string, default '_' If appending prefix, separator/delimiter to use. Or pass a list or dictionary as with `prefix.` dummy_na : bool, default False Add a column to indicate NaNs, if False NaNs are ignored. columns : list-like, default None Column names in the DataFrame to be encoded. If `columns` is None then all the columns with `category` dtype will be converted. sparse : bool, default False Whether the dummy columns should be sparse or not. Returns SparseDataFrame if `data` is a Series or if all columns are included. Otherwise returns a DataFrame with some SparseBlocks. .. versionadded:: 0.18.2 drop_first : bool, default False Whether to get k-1 dummies out of k categorical levels by removing the first level. dtype : dtype, default np.uint8 Data type for new columns. Only a single dtype is allowed. Only valid if pandas is 0.23.0 or newer. .. versionadded:: 0.18.2 Returns ------- dummies : DataFrame Examples -------- Dask's version only works with Categorical data, as this is the only way to know the output shape without computing all the data. >>> import pandas as pd >>> import dask.dataframe as dd >>> s = dd.from_pandas(pd.Series(list('abca')), npartitions=2) >>> dd.get_dummies(s) Traceback (most recent call last): ... NotImplementedError: `get_dummies` with non-categorical dtypes is not supported... With categorical data: >>> s = dd.from_pandas(pd.Series(list('abca'), dtype='category'), npartitions=2) >>> dd.get_dummies(s) # doctest: +NORMALIZE_WHITESPACE Dask DataFrame Structure: a b c npartitions=2 0 uint8 uint8 uint8 2 ... ... ... 3 ... ... ... Dask Name: get_dummies, 4 tasks >>> dd.get_dummies(s).compute() a b c 0 1 0 0 1 0 1 0 2 0 0 1 3 1 0 0 See Also -------- pandas.get_dummies z0.23.0dtypezOYour version of pandas is '{}'. The 'dtype' keyword was added in pandas 0.23.0.)prefix prefix_sepdummy_nacolumnssparse drop_firstz`get_dummies` with non-categorical dtypes is not supported. Please use `df.categorize()` beforehand to convert to categorical dtype.z`get_dummies` with unknown categories is not supported. Please use `column.cat.as_known()` or `df.categorize()` beforehand to ensure known categoriesNobjectcategory)Zincludec3s|]}t|VqdS)N)r ).0c)data5lib/python3.7/site-packages/dask/dataframe/reshape.py szget_dummies..c3s|]}t|VqdS)N)r )rr)rrrrs)rrrrrrmeta)rnpuint8 ValueErrorformat isinstancepdrr get_dummiesr NotImplementedErrorr ZdtypesanyZ_metaZ select_dtypesrallr) rrrrrrrrkwargsZ not_cat_msgZunknown_cat_msgrr)rrr&sJS          r&meanc Cs,t|r|dkrtdt|r(|dkr0tdt||sDtdt||sXtdt|rh|dkrptdt|r|dkrtdtj||jj|d }tj|t j d }||j _ |||d }t |gtjtj|d |d }t |gtjtj|d|d } |dkr|S|dkr| S|dkr$|| StdS)a Create a spreadsheet-style pivot table as a DataFrame. Target ``columns`` must have category dtype to infer result's ``columns``. ``index``, ``columns``, ``values`` and ``aggfunc`` must be all scalar. Parameters ---------- data : DataFrame values : scalar column to aggregate index : scalar column to be index columns : scalar column to be columns aggfunc : {'mean', 'sum', 'count'}, default 'mean' Returns ------- table : DataFrame Nz.'index' must be the name of an existing columnz0'columns' must be the name of an existing columnz 'columns' must be category dtypezs'columns' must have known categories. Please use `df[columns].cat.as_known()` beforehand to ensure known categoriesz/'values' must be the name of an existing column)r+sumcountz/aggfunc must be either 'mean', 'sum' or 'count')name)rr)indexrvaluesZpivot_table_sum)chunkZ aggregatertokenZ chunk_kwargsZpivot_table_countr,r-r+)r r"r r r%ZCategoricalIndexcatZ categoriesrr Zfloat64r/r.r r Z pivot_sumZ pivot_aggZ pivot_count) Zdfr/rr0ZaggfuncZ new_columnsrr*Zpv_sumZpv_countrrr pivot_tablesF      r4valuec Cs(ddlm}|jtj||||||ddS)Nr) no_defaultmelt)rid_vars value_varsvar_name value_name col_levelr2)Zdask.dataframe.corer6rr%r7)framer8r9r:r;r<r6rrrr7s   r7)NNNr+)NNNr5N)Z __future__rrrZnumpyr Zpandasr%Zcorerrrr r Zutilsr r r rr!r&r4r7rrrrs   I