B ZE@s dZddlmZmZmZmZmZmZddlZ ddl m m Z ddlmZddlmZddlZddlmZddlmZmZddlmZd-dd Zd.d d Zd/ddZd0ddZddZ d1ddZ!ddZ"ddZ#ddZ$d2dd Z%d!d"Z&d#d$Z'd%d&Z(d'd(Z)Gd)d*d*e*Z+d3d+d,Z,dS)4z Utility functions models code )reducelziplmapasstr2rangelongN)svdvals)webuse)_is_using_pandas _is_recarray)np_matrix_rankcCs0i}x&t|D]\}}||||iqW|S)zd Helper function to create a dictionary mapping a column number to the name in tmp_arr. ) enumerateupdate)tmp_arroffsetcol_mapiZcol_namer6lib/python3.7/site-packages/statsmodels/tools/tools.py_make_dictnamessrcCst|}|jdkr$|dddf}|dk rt|}|jdkrP|dddf}tt||t||}||||fSt||}||SdS)a Returns views on the arrays Y and X where missing observations are dropped. Y : array-like X : array-like, optional axis : int Axis along which to look for missing observations. Default is 1, ie., observations in rows. Returns ------- Y : array All Y where the X : array Notes ----- If either Y or X is 1d, it is reshaped to be 2d. rN)npasarrayndimarrayZ logical_andisnanany)YXaxisZkeepidxrrr drop_missings    r Fc stttfr@ytdks tdWntdYnX|jjsV|jt j krPsrt |j dkrrt dtttfr|jjdkr|jjrt|jjdkr|jjdt |}d}|j dkr|dddf}d}||kt}|rt |dd}|jjs6dd t |D}n"|jjrXd d t |D}dkry|jjdWnd YnXfd d |D}|dkr.t|jdkr|jd|jdkrt |dd}t||jjgt|}t jtt||d t|Stj|dt|t j kd}tj|||dt|t j kd}|S|jt jkslt|t jst|t jst dtttfr<|jd}t |ddf}|ddt j!f|ddfkt}|dd}|dkr|d8}t j"|ddt}t #||f}|dkr8t$||} || fS|Sdkrt |j dkrt |}|dddf|kt}|dd}|dkr|dkrt$|} || fS|St #||f}|dkrt$|dd} || fS|Sn t ddS)a% Returns a dummy matrix given an array of categorical variables. Parameters ---------- data : array A structured array, recarray, or array. This can be either a 1d vector of the categorical variable or a 2d array with the column specifying the categorical variable specified by the col argument. col : 'string', int, or None If data is a structured array or a recarray, `col` can be a string that is the name of the column that contains the variable. For all arrays `col` can be an int that is the (zero-based) column index number. `col` can only be None for a 1d array. The default is None. dictnames : bool, optional If True, a dictionary mapping the column number to the categorical name is returned. Used to have information about plain arrays. drop : bool Whether or not keep the categorical variable in the returned matrix. Returns -------- dummy_matrix, [dictnames, optional] A matrix of dummy (indicator/binary) float variables for the categorical data. If dictnames is True, then the dictionary is returned as well. Notes ----- This returns a dummy variable for EVERY distinct variable. If a a structured or recarray is provided, the names for the new variable is the old variable name - underscore - category name. So if the a variable 'vote' had answers as 'yes' or 'no' then the returned array would have to new variables-- 'vote_yes' and 'vote_no'. There is currently no name checking. Examples -------- >>> import numpy as np >>> import statsmodels.api as sm Univariate examples >>> import string >>> string_var = [string.ascii_lowercase[0:5], string.ascii_lowercase[5:10], string.ascii_lowercase[10:15], string.ascii_lowercase[15:20], string.ascii_lowercase[20:25]] >>> string_var *= 5 >>> string_var = np.asarray(sorted(string_var)) >>> design = sm.tools.categorical(string_var, drop=True) Or for a numerical categorical variable >>> instr = np.floor(np.arange(10,60, step=2)/10) >>> design = sm.tools.categorical(instr, drop=True) With a structured array >>> num = np.random.randn(25,2) >>> struct_ar = np.zeros((25,1), dtype=[('var1', 'f4'),('var2', 'f4'), ('instrument','f4'),('str_instr','a5')]) >>> struct_ar['var1'] = num[:,0][:,None] >>> struct_ar['var2'] = num[:,1][:,None] >>> struct_ar['instrument'] = instr[:,None] >>> struct_ar['str_instr'] = string_var[:,None] >>> design = sm.tools.categorical(struct_ar, col='instrument', drop=True) Or >>> design2 = sm.tools.categorical(struct_ar, col='str_instr', drop=True) rrz%Can only convert one column at a timez)col is None and the input array is not 1dNTFcSsg|] }t|qSr)r).0itemrrr szcategorical..cSsg|] }t|qSr)r)r!r"rrrr#svarcsg|]}d|qS)_r)r!r")colrrr#s)dtype)usemask asrecarray)datar(r)z$Array-like objects are not supported)r)rzThe index %s is not understood)% isinstancelisttuplelenAssertionError ValueErrorr'names __class__rZrecarrayZsqueezer IndexErrorintruniqueastypefloatZswapaxestolistshaperstrrrZviewtypenprfZ drop_fieldsZ append_fieldsndarrayNotImplementedErrornewaxisdelete column_stackr) r*r&Z dictnamesZdroprZ_swapZ tmp_dummyZdtrrr)r&r categoricalBsK            (          rBTskipcCst|dst|r.ddlm}||d||dSt|}|jdkrT|dddf}n|jdkrftdtj|dd dk}|tj |d kdd M}| r|d kr|S|d krtd t |j d|g}|r|n |ddd}t |S)a Adds a column of ones to an array Parameters ---------- data : array-like ``data`` is the column-ordered design matrix prepend : bool If true, the constant is in the first column. Else the constant is appended (last column). has_constant : str {'raise', 'add', 'skip'} Behavior if ``data`` already has a constant. The default will return data without adding another constant. If 'raise', will raise an error if a constant is present. Using 'add' will duplicate the constant, if one is present. Returns ------- data : array, recarray or DataFrame The original values with a constant (column of ones) as the first or last column. Returned value depends on input type. Notes ----- When the input is recarray or a pandas Series or DataFrame, the added column's name is 'const'. Nr) add_trendc)Ztrendprepend has_constantrz$Only implementd 2-dimensional arrays)rgrCraisez data already contains a constant)r r Zstatsmodels.tsa.tsatoolsrDrZ asanyarrayrr0ZptpallrZonesr9rA)r*rFrGrDxZis_nonzero_constrrr add_constants$    rMcCszt|}t|}|jdkr.|dddf}|jd|jdkrTtd|jdt||g}t|t|krvdSdS)a  True if (Q, P) contrast `C` is estimable for (N, P) design `D` From an Q x P contrast matrix `C` and an N x P design matrix `D`, checks if the contrast `C` is estimable by looking at the rank of ``vstack([C,D])`` and verifying it is the same as the rank of `D`. Parameters ---------- C : (Q, P) array-like contrast matrix. If `C` has is 1 dimensional assume shape (1, P) D: (N, P) array-like design matrix Returns ------- tf : bool True if the contrast `C` is estimable on design `D` Examples -------- >>> D = np.array([[1, 1, 1, 0, 0, 0], ... [0, 0, 0, 1, 1, 1], ... [1, 1, 1, 1, 1, 1]]).T >>> isestimable([1, 0, 0], D) False >>> isestimable([1, -1, 0], D) True rNzContrast should have %d columnsFT)rrrr9r0Zvstackr )CDnewrrr isestimable$s   rQV瞯.)raddrZ flatnonzerorrY)riZcolsumvalr)rirclean0srlcCs|dkrt|}tj|dd\}}}t|}|ddd}g}x*t|D]}||dd||fqJWtt| tj S)z Return a matrix whose column span is the same as X. If the rank of X is known it can be specified as r -- no check is made to ensure that this really is the rank of X. Nr)Z full_matricesrJ) r LrUrZargsortrappendrrYr6ra)rrVrOUordervaluerrrrfullranks  rtcCst|}d||<||S)a Unsqueeze a collapsed array >>> from numpy import mean >>> from numpy.random import standard_normal >>> x = standard_normal((3,4,5)) >>> m = mean(x, axis=1) >>> m.shape (3, 5) >>> m = unsqueeze(m, 1, x.shape) >>> m.shape (3, 1, 5) >>> r)r,Zreshape)r*rZoldshapeZnewshaperrr unsqueezesrucGstdd|dddS)a Returns the dot product of the given matrices. Parameters ---------- arrs: argument list of ndarray Returns ------- Dot product of all arguments. Examples -------- >>> import numpy as np >>> from statsmodels.tools import chain_dot >>> A = np.arange(1,13).reshape(3,4) >>> B = np.arange(3,15).reshape(4,3) >>> C = np.arange(5,8).reshape(3,1) >>> chain_dot(A,B,C) array([[1820], [4300], [6780]]) cSs t||S)N)rrX)rLyrrrszchain_dot..NrJ)r)Zarrsrrr chain_dotsrxcCsZtt||dk}t|dkt|}||}tt|t|}tj||<|S)z Returns np.dot(left_matrix, right_matrix) with the convention that nan * 0 = 0 and nan * x = nan if x != 0. Parameters ---------- A, B : np.ndarrays r)rrXrZ nan_to_numrc)ABZshould_be_nan_1Zshould_be_nan_2Z should_be_nanrNrrrnan_dots  r{cCs t|d|S)z Gets raw results back from wrapped results. Can be used in plotting functions or other post-estimation type routines. Z_results)getattr)Zresultsrrrmaybe_unwrap_resultssr}cs eZdZdZfddZZS)BunchzO Returns a dict-like object with keys accessible via attribute lookup. cstt|j||||_dS)N)superr~__init____dict__)selfargskwargs)r2rrrszBunch.__init__)__name__ __module__ __qualname____doc__r __classcell__rr)r2rr~sr~cCs|dkr |St|d}|jdkr8|r.||jfS|dfSn|jdkrJtd|rT|jnd}|rvt|dddf|fSt||fSdS)a Parameters ---------- x : array, Series, DataFrame or None Input to verify dimensions, and to transform as necesary ndarray : bool Flag indicating whether to always return a NumPy array. Setting False will return an pandas DataFrame when the input is a Series or a DataFrame. Returns ------- out : array, DataFrame or None array or DataFrame with 2 dimensiona. One dimensional arrays are returned as nobs by 1. None is returned if x is None. names : list of str or None list containing variables names when the input is a pandas datatype. Returns None if the input is an ndarray. Notes ----- Accepts None for simplicity NrHzx mst be 1 or 2-dimensional.) r rcolumnsr0namerrpdZ DataFrame)rLr=Z is_pandasrrrr _ensure_2ds     r)r)Nr)NFF)TrC)rR)N)F)-rZstatsmodels.compat.pythonrrrrrrZnumpyrZnumpy.lib.recfunctionslibZ recfunctionsr<Z numpy.linalgrTrmZ scipy.linalgrZpandasrZstatsmodels.datasetsr Zstatsmodels.tools.datar r Zstatsmodels.compat.numpyr rr rBrMrQr_rgrhrlrtrurxr{r}dictr~rrrrrs4      ' / 4)