B Z_ @s`ddlmZmZmZmZmZddlmZddlZ ddl m m Z ddlZddlmZddlmZddlmZddlmZddlmZmZdddZd?ddZd@ddZddZddZ ddZ!d d!Z"d"d#Z#d$d%Z$d&d'Z%d(d)Z&d*d+Z'd,d-Z(d.d/Z)d0d1Z*d2d3Z+d4d5Z,d6d7Z-d8d9Z.d:d;Z/ddd d)d+d-ddd%d'g Z0dS)A)rangelrangelziplongPY3)recarray_selectN) DataFrame)offsets) to_offset) ValueWarning)_is_using_pandas _is_recarraycFskipcCs|}dddg}|dkr,|dd}d}nP|dks<|d krb|dd }|d kr\|dd }d}n|d krpd }n td |t|}t|dp|}|s|r|r|jj}tj|}qt |tj rt|}q| }n t |}t|} t t jd| dt jd |d} t | } |d kr.| dddf} d|kr|sD|rZdd} || d} n8t t jt jt |dddkddt j|dkdd} t | r|dkrtdn*|dkr|dd}| ddddf} |rdnd} |s|r(tj| |j|d} | |g}t|dd| d}n| |g}t |dd| }|r|jddd}|jj}t|t|}|r|d||}n||| d}tsdd|D}dd|D}dd|D}tt||}|t |}|S)a Adds a trend and/or constant to an array. Parameters ---------- X : array-like Original array of data. trend : str {"c","t","ct","ctt"} "c" add constant only "t" add trend only "ct" add constant and linear trend "ctt" add constant and linear and quadratic trend. prepend : bool If True, prepends the new data to the columns of X. has_constant : str {'raise', 'add', 'skip'} Controls what happens when trend is 'c' and a constant already exists in X. 'raise' will raise an error. 'add' will duplicate a constant. 'skip' will return the data without change. 'skip' is the default. Returns ------- y : array, recarray or DataFrame The original data with the additional trend columns. If x is a recarray or pandas Series or DataFrame, then the trend column names are 'const', 'trend' and 'trend_squared'. Notes ----- Returns columns as ["ctt","ct","c"] whenever applicable. There is currently no checking for an existing trend. See also -------- statsmodels.tools.tools.add_constant ZconsttrendZ trend_squaredrNrZcttZcttztrend %s not understood)dtypecSs,yt|dkot|dkSdSdS)NgF)npptpany)sr7lib/python3.7/site-packages/statsmodels/tsa/tsatools.py safe_is_const\sz add_trend..safe_is_const)axisgraisezx already contains a constantr)indexcolumnsF)rZconvert_datetime64cSsg|] }|dqS)rr).0entryrrr szadd_trend..cSsg|] }|dqS)rr)r!r"rrrr#scSsg|] }t|qSr)bytes)r!namerrrr#s) lower ValueErrorr r rdescrpdrZ from_records isinstanceZSeriescopyr asanyarraylenvanderarangeZfloat64ZfliplrZapplyZ logical_andrrallrconcat column_stackZ to_recordsrlistzipZastype)xrZprependZ has_constantr Z trendorderZ is_recarray is_pandasr(nobsZtrendarrrZ col_constorderZ new_descrZ extra_colnamesZdtypesrrr add_trendsz&         "    $      r:rTcsfjjrjj}s0tjdkr0tdnt|dkrD|dttt fr^jjt sdd|D}|j_tt rt }fddt ddD}t|dd}|d krt|d} n<|d krt|d} n&|t|krdd l} | d t|} t|d | } t|| d } |rf| krV| | n| | | rt| } tj| d ||jd d }nTtjtt|jfd}x*t|D]\}}|d d |f||<qW| rtj|| fdd| Dd d S|SnJjdkr4d d d fd krBddkrZjdd d f}|d kr~d} nd|d krjd} nN|dkrjd|d}|jdkrވjd}dd l} | d t|} t|dd}t| }t| jd}|r:|kr*||n||td |f|d |ffSd S)a Returns an array with lags included given an array. Parameters ---------- x : array An array or NumPy ndarray subclass. Can be either a 1d or 2d array with observations in columns. col : 'string', int, or None If data is a structured array or a recarray, `col` can be a string that is the name of the column containing the variable. Or `col` can be an int of the zero-based column index. If it's a 1d array `col` can be None. lags : int The number of lags desired. drop : bool Whether to keep the contemporaneous variable for the data. insert : bool or int If True, inserts the lagged values after `col`. If False, appends the data. If int inserts the lags at int. Returns ------- array : ndarray Array with lags Examples -------- >>> import statsmodels.api as sm >>> data = sm.datasets.macrodata.load() >>> data = data.data[['year','quarter','realgdp','cpi']] >>> data = sm.tsa.add_lag(data, 'realgdp', lags=2) Notes ----- Trims the array both forward and backward, so that the array returned so that the length of the returned array is len(`X`) - lags. The lags are returned in increasing order, ie., t-1,t-2,...,t-lags rz)col is None and the input array is not 1drcSs"g|]}t|trt|n|qSr)r*unicoder$)r!r%rrrr#szadd_lag..csg|]}dd|qS)_zL(%i)r)r!i)colrrr#sZBoth)maxlagtrimTFNz number of variables, inserting at the last position)Zusemask)rcsg|]}|dqS)Nr)r!r%)lagsr5rrr#s)r@)rr9rZsqueezendim IndexErrorr-r*intrrr;r$rlagmatr3rwarningswarnr poprnprfZ append_fieldsTzerosr enumerateshaperr2)r5r>rAdropinsertr9ZcontempZ tmp_namesZndlagsZins_idxrFZ first_namesZ last_namesZ_xZ first_arrr=r%Z first_colsZ last_colsr)r>rAr5radd_lags)                  rPcCs|jdkrt|dkr|j}n|jdkr0td|jd}|dkrT||jdd}n>tjtt ||dd}tj | |}|t ||}|jdkrt|dkr|j}|S)a Detrend an array with a trend of given order along axis 0 or 1 Parameters ---------- x : array_like, 1d or 2d data, if 2d, then each row or column is independently detrended with the same trendorder, but independent trend estimates order : int specifies the polynomial order of the trend, zero is constant, one is linear trend, two is quadratic trend axis : int axis can be either 0, observations by rows, or 1, observations by columns Returns ------- detrended data series : ndarray The detrended series is the residual of the linear regression of the data on the trend of given order. rrz0x.ndim > 2 is not implemented until it is neededr)r)N) rBrDrJNotImplementedErrorrMZmeanrr.r/floatZlinalgZpinvdot)r5r8rr7ZresidZtrendsZbetarrrdetrends  rUforwardexcs*t|do |}|dkrdn|}|}|r:|dkr:tdt|}d}|jdkrb|dddf}|j\}} |dkrx| }||krtdt||| |df} xNtdt |dD]8} || || ||| | || | || df<qW|d krd} n|d kr|} ntd |dkr.t | } n|} |rt |t rJ|j n|jg}d d |D}x6t|D]*}t|d|fdd |DqjWt | d| |j|d} | j| d}|dkr||}||d}n2| | | |df}|dkr| | | d|f}|dkr"||fS|SdS)a Create 2d array of lags Parameters ---------- x : array_like, 1d or 2d data; if 2d, observation in rows and variables in columns maxlag : int all lags from zero to maxlag are included trim : str {'forward', 'backward', 'both', 'none'} or None * 'forward' : trim invalid observations in front * 'backward' : trim invalid initial observations * 'both' : trim invalid observations on both sides * 'none', None : no trimming of observations original : str {'ex','sep','in'} * 'ex' : drops the original array returning only the lagged values. * 'in' : returns the original array and the lagged values as a single array. * 'sep' : returns a tuple (original array, lagged values). The original array is truncated to have the same number of rows as the returned lagmat. use_pandas : bool, optional If true, returns a DataFrame when the input is a pandas Series or DataFrame. If false, return numpy ndarrays. Returns ------- lagmat : 2d array array with lagged observations y : 2d array, optional Only returned if original == 'sep' Examples -------- >>> from statsmodels.tsa.tsatools import lagmat >>> import numpy as np >>> X = np.arange(1,7).reshape(-1,2) >>> lagmat(X, maxlag=2, trim="forward", original='in') array([[ 1., 2., 0., 0., 0., 0.], [ 3., 4., 1., 2., 0., 0.], [ 5., 6., 3., 4., 1., 2.]]) >>> lagmat(X, maxlag=2, trim="backward", original='in') array([[ 5., 6., 3., 4., 1., 2.], [ 0., 0., 5., 6., 3., 4.], [ 0., 0., 0., 0., 5., 6.]]) >>> lagmat(X, maxlag=2, trim="both", original='in') array([[ 5., 6., 3., 4., 1., 2.]]) >>> lagmat(X, maxlag=2, trim="none", original='in') array([[ 1., 2., 0., 0., 0., 0.], [ 3., 4., 1., 2., 0., 0.], [ 5., 6., 3., 4., 1., 2.], [ 0., 0., 5., 6., 3., 4.], [ 0., 0., 0., 0., 5., 6.]]) Notes ----- When using a pandas DataFrame or Series with use_pandas=True, trim can only be 'forward' or 'both' since it is not possible to consistently extend index values. Nnone)rXbackwardzDtrim cannot be 'none' or 'forward' when used on Series or DataFramesrr)rWsepzmaxlag should be < nobs)rXrV)rYZbothztrim option not validcSsg|] }t|qSr)str)r!r>rrrr#szlagmat..csg|]}t|dqS)z.L.)r[)r!r>)lag_strrrr#s)rr )rZrWrZ)r r&r'rasarrayrBrMrKrrDr-r*rr r%r[extendrilocrN)r5r?r@original use_pandasr6ZxaZdropidxr7nvarZlmkZstartobsZstopobsZ x_columnsr ZlagrAZleadsr)r\rrEAsTA    8        rEc Cs|dkr |}t||}t|d}|jdkrL|r:t|}qh|dddf}n|jdks`|jdkrhtd|j\}} |r|rt|jdddf||ddd} | jddd|dfg} xRt d| D]D} t|jdd| f||ddd} | | jdd||dfqWtj | dd S|r,t |}t|dddf||dd ddd|dfg} xJt d| D]<} | t|dd| f||dd dd||dfqjWt | S) a Generate lagmatrix for 2d array, columns arranged by variables Parameters ---------- x : array_like, 2d 2d data, observation in rows and variables in columns maxlag0 : int for first variable all lags from zero to maxlag are included maxlagex : None or int max lag for all other variables all lags from zero to maxlag are included dropex : int (default is 0) exclude first dropex lags from other variables for all variables, except the first, lags from dropex to maxlagex are included trim : string * 'forward' : trim invalid observations in front * 'backward' : trim invalid initial observations * 'both' : trim invalid observations on both sides * 'none' : no trimming of observations use_pandas : bool, optional If true, returns a DataFrame when the input is a pandas Series or DataFrame. If false, return numpy ndarrays. Returns ------- lagmat : 2d array array with lagged observations, columns ordered by variable Notes ----- Inefficient implementation for unequal lags, implemented for convenience Nrrrz'Only supports 1 and 2-dimensional data.inT)r@r`ra)r)r@r`)maxr rBr)r TypeErrorrMrEr_rappendr1rr,r2) r5Zmaxlag0ZmaxlagexZdropexr@rar?r6r7rbrAZlagslircrrr lagmat2dss4$        $ 2<rhcCs |dS)NF)ravel)matrrrvecsrlcCs|jtt|S)N)rJtake _triu_indicesr-)rkrrrvechsrocCst|\}}|||S)N)rZ tril_indices)nrowscolsrrr _tril_indicessrscCst|\}}|||S)N)r triu_indices)rprqrrrrrrn srncCst|\}}|||S)N)r diag_indices)rprqrrrrr _diag_indicessrvcCs8ttt|}||t|ks&t|j||fddS)Nri)r8)rDrsqrtr-AssertionErrorreshape)vrcrrrunvecsr{cCslddtddt|}tt|}t||f}||t|<||j}|t|d<|S)Ng?rrr) rrwr-rDroundrKrtrJru)rzrqresultrrrunvechs rcCs,t||dd}tdd|DjS)z Create duplication matrix D_n which satisfies vec(S) = D_n vech(S) for symmetric matrix S Returns ------- D_n : ndarray rrcSsg|]}t|qSr)rrj)r!r5rrrr#/sz&duplication_matrix..)reyeZarrayrJ)rptmprrrduplication_matrix%s rcCs.ttt||f}t|||dkS)z Create the elimination matrix L_n which satisfies vech(M) = L_n vec(M) for any matrix M Parameters ---------- Returns ------- r)rlrZtrilZonesr)rpZ vech_indicesrrrelimination_matrix1s rcCs<t||}t||j||fdd}|j|ddS)z Create the commutation matrix K_{p,q} satisfying vec(A') = K_{p,q} vec(A) Parameters ---------- p : int q : int Returns ------- K : ndarray (pq x pq) ri)r8r)r)rrr/ryrmrj)pqKindicesrrrcommutation_matrix@s rc Csdt| dt| }dt| dt| }xdtdt|D]R}||}x0t|D]$}||||||d8<qnW|d||d|<qXW|S)z Transforms params to induce stationarity/invertability. Parameters ---------- params : array The AR coefficients Reference --------- Jones(1980) rN)rexpr+rr-)params newparamsrjakiterrrr_ar_transparamsQs $rcCs|}xvtt|dddD]^}||}x.rr)rr]r)r5rr)rr5runintegrate_levelss rcCs\t|dd}t|dkr@|d}tttj||f|S|d}ttj||fS)ay After taking n-differences of a series, return the original series Parameters ---------- x : array-like The n-th differenced series levels : list A list of the first-value in each differenced series, for [first-difference, second-difference, ..., n-th difference] Returns ------- y : array-like The original series de-differenced Examples -------- >>> x = np.array([1, 3, 9., 19, 8.]) >>> levels = unintegrate_levels(x, 2) >>> levels array([ 1., 2.]) >>> unintegrate(np.diff(x, 2), levels) array([ 1., 3., 9., 19., 8.]) Nrrr)r3r-rH unintegraterZcumsumZr_)r5ZlevelsZx0rrrrs   rcCst|tjst|}|j}|dks0|dr4dS|dksF|drJdS|dks\|dr`d S|d ksr|d rvd S|d krdS|dkrdS|dkrdStd|dS)a- Convert a pandas frequency to a periodicity Parameters ---------- freq : str or offset Frequency to convert Returns ------- period : int Periodicity of freq Notes ----- Annual maps to 1, quarterly maps to 4, monthly to 12, weekly to 52. A)zA-zAS-rQ)zQ-zQS-M)zM-ZMS WzW-4DBHzDfreq {} not understood. Please report if you think this is in error.N) r*r Z DateOffsetr Z rule_codeupper startswithr'format)Zfreqrrrfreq_to_periods&  r)rFr)NrFT)rr)rVrWF)NrrVF)1Zstatsmodels.compat.pythonrrrrrZstatsmodels.compat.numpyrZnumpyrZnumpy.lib.recfunctionslibZ recfunctionsrIZpandasr)rZpandas.tseriesr Zpandas.tseries.frequenciesr Zstatsmodels.tools.sm_exceptionsr Zstatsmodels.tools.datar r r:rPrUrErhrlrorsrnrvr{rrrrrrrrrrr__all__rrrrsH      {  * x C ")