B ¥ŠãZf‡ã @s¤ddlmZddlmZmZmZmZmZddlZ ddlm Z m Z ddl m Z mZddlmZddlmZddlmZmZmZmZddlmmmZddlmmZdd lm Z m!Z!m"Z"dd l#m$Z$m%Z%dd l&m'Z'ddl(mm)Z*dd l+m,Z,d gZ-d4dd„Z.dd„Z/d5dd„Z0Gdd „d ej1ƒZ2Gdd„dej3ƒZ4Gdd„de*j5ƒZ6e* 7e6e4¡e8dkr ddl9m:Z;e;j¡Z=e2e=j?ƒZ@e@jAddZBe2e=j?ƒZCeCjAdddddd ZDddlEZEddlFmGZHeHjId!d"d#ZJeHjKeJeLe=j?ƒd$ZMeEjNeJjOeLe=j?ƒd%d&ZPe  Qd!d!eLe=j?ƒ¡ZReHjKeRd"d'ZRddlOZOe  SeeOjOjTeM U¡ VeW¡ƒ¡ZXeEjYe=j?eXd(Z=e2e=d"d'ZZeZjAddd)Z[e  Sd*d+d,d-d.d-d/d0d1d2g ¡Z\e  ]e\¡Z^d3Z_dS)6é)Údivision)Ú iteritemsÚrangeÚ string_typesÚlmapÚlongN)ÚdotÚidentity)ÚinvÚslogdet)Únorm)ÚOLS)ÚlagmatÚ add_trendÚ_ar_transparamsÚ_ar_invtransparams)Úresettable_cacheÚcache_readonlyÚcache_writable)Ú approx_fprimeÚ approx_hess)Ú KalmanFilter)ÚutilÚARcCstj|d|dS)z= k_ar for conditional MLE or dynamic forecast. Got %d)Ú ValueError)ÚstartÚk_arÚmethodÚdynamicrrr Ú_check_ar_startsr(c Cs¸|d|…pd}||d…ddd…}t ||¡}|rR||||…|d|…<n|| d…|d|…<t |¡} x@t|ƒD]4} |t ||| | |…¡} | | | <| || |<q|W| S)Nréÿÿÿÿ)rÚzerosrr) ÚyÚparamsÚpÚk_trendZstepsr$ÚmuZarparamsÚendogZforecastÚiZfcastrrr Ú_ar_predict_out_of_sample$s r2c sÄeZdZejddejddœZd1‡fdd„ Zd d „Z d d „Z d d„Z dd„Z d2‡fdd„ Z d3dd„Zdd„Zdd„Zdd„Zdd„Zdd„Zd d!„Zd"d#„Zd$d%„Zd4d(d)„Zd5‡fd/d0„ Z‡ZS)6rzAutoregressive AR(p) modelzVendog : array-like 1-d endogenous response variable. The independent variable.Ú)Úmodelr,Z extra_paramsZextra_sectionsNÚnonecsftt|ƒj|d|||d|j}|jdkrB|dd…df}||_n |jdkrb|jddkrbtdƒ‚dS)N)Úmissingéz'Only the univariate case is implemented)ÚsuperrÚ__init__r0ÚndimÚshaper#)Úselfr0ÚdatesÚfreqr6)Ú __class__rr r9?s z AR.__init__cCsdS)Nr)r<rrr Ú initializeHsz AR.initializecCs<|j}|j}| ¡}t||||… ¡ƒ||||…<|S)z‚ Transforms params to induce stationarity/invertability. Reference --------- Jones(1980) )r%r.Úcopyr)r<r,r-ÚkÚ newparamsrrr Ú _transparamsKs $zAR._transparamscCs<|j}|j}| ¡}t||||… ¡ƒ||||…<|S)z9 Inverse of the Jones reparameterization )r%r.rAr)r<Ú start_paramsr-rBrCrrr Ú_invtransparamsYs $zAR._invtransparamscCs@|j}t ||||¡}t |||d|¡} t |df¡} ttt|dƒt  ||¡ƒt| | jƒ  d¡ƒ} | j ||dd} | } t  |¡} x¶t |ƒD]ª}||t| | ƒ}tt| | ƒ| jƒ}d|}ttt|| ƒ| jƒ|ƒ}t|| ƒt||ƒ} |t|| ƒ}tt|| ƒ|jƒt| | jƒ} ||dkrŽt| | ƒ||d|<qŽWdS)z¤ Return the pre-sample predicted values using the Kalman Filter Notes ----- See predict method for how to use start and p. rr7rÚF)Úordergð?N)r.rÚTÚRrr*rr r ZkronZravelZreshapeÚZr)r<r,r$r-Úendr+ÚpredictedvaluesrBZT_matZR_matZalphaZQ_0ÚPZZ_matr1Zv_matZF_matZFinvÚKÚLrrr Ú_presample_fitcs&  zAR._presample_fitc s¤t|ddƒ}t|ddƒ}|dkr@|dkr2|s2d}n|}|j|}|dkrR|jd}tt|ƒ |||¡\}}}}d|krŠ||krŠtd|ƒ‚t||||ƒ||||fS)Nr&Úmler%rr)zEStart must be >= k_ar for conditional MLE or dynamic forecast. Got %s)ÚgetattrZ_indexr8rÚ_get_prediction_indexr#r() r<r$rLr'Úindexr&r%Ú out_of_sampleZprediction_index)r?rr rT†s     zAR._get_prediction_indexFc Cs€| |||¡\}}}}||dkr*tdƒ‚|j}|j}|j} |j ¡} |rl|||d7}t| |||||ƒSt  |d|¡} | dkrø|rª|ddt  ||d…¡} nd} ||krø|  |||t |d|ƒ| d|…| | ¡| d||…| 7<||kr| St |j|ƒ} t||dƒ}t||dƒ}t t| ƒ||dƒ}| ||…| |d…<|r|t| ||||ƒ}tj| |f} | S)aò Returns in-sample and out-of-sample prediction. Parameters ---------- params : array The fitted model parameters. start : int, str, or datetime Zero-indexed observation number at which to start forecasting, ie., the first forecast is start. Can also be a date string to parse or a datetime type. end : int, str, or datetime Zero-indexed observation number at which to end forecasting, ie., the first forecast is start. Can also be a date string to parse or a datetime type. dynamic : bool The `dynamic` keyword affects in-sample prediction. If dynamic is False, then the in-sample lagged values are used for prediction. If `dynamic` is True, then in-sample forecasts are used in place of lagged dependent variables. The first forecasted value is `start`. Returns ------- predicted values : array Notes ----- The linear Gaussian Kalman filter is used to return pre-sample fitted values. The exact initial Kalman Filter is used. See Durbin and Koopman in the references for more information. r7zend is before startrRrN)rTr#r%r.r&r0Úsqueezer2rr*rrQÚminrÚXÚmaxÚlenÚr_)r<r,r$rLr'rVÚ_r%r.r&r0rMr/Ú fittedvaluesZpv_startZfv_startZfv_endZforecastvaluesrrr ÚpredictžsB#      z AR.predictc CsØ|j}|j}|d}tjd||d…f}tj||f|jd}x|td|ƒD]n}t ||d|…¡dd…||d|dd…f<||d|dd…ft || d…|¡dd…8<qJW||jt  |  ¡¡}|S)z„ Returns the inverse of the presample variance-covariance. Notes ----- See Hamilton p. 125 r7r)N)Údtype) r.r%rr\r*r`rZ correlaterIÚdiagZdiagonal)r<r,rBr-Zp1Zparams0ÚVpinvr1rrr Ú_presample_varcovòs0>zAR._presample_varcovcCsb|j}|j}|j}t| ¡t ||¡ƒ}||}| dt dtj¡t |¡|d|S)zQ Loglikelihood of AR(p) process using conditional sum of squares r) ÚnobsÚYrYr!rWrrÚlogÚpi)r<r,rdrerYÚssrÚsigma2rrr Ú _loglike_css s"zAR._loglike_cssc Cs,|j}|j}|j}|j}|j}|jr.| |¡}|d|… ¡}|rR|dg|}ndg}t  |dt  ||d…¡¡} || dd…df} |  |¡} t  t  | j | ¡| ¡ ¡} t||d… ¡t  ||¡ƒ} d|| | }||_t| ƒd}d|t dtj¡t |¡|| || |}|S)zO Loglikelihood of AR(p) process using exact maximum likelihood Nrr7gð?gà¿r)rdrYr0r%r.Ú transparamsrDrArÚasarrayrrcrrIÚitemr!rWrir rfrg)r<r,rdrYr0r%r.ZypÚcZmupZdiffprbZ diffpVpinvrhriZlogdetÚloglikerrr Ú _loglike_mles,     4zAR._loglike_mlecCs"|jdkr| |¡S| |¡SdS)a¡ The loglikelihood of an AR(p) process Parameters ---------- params : array The fitted parameters of the AR model Returns ------- llf : float The loglikelihood evaluated at `params` Notes ----- Contains constant term. If the model is fit by OLS then this returns the conditonal maximum likelihood. .. math:: \frac{\left(n-p\right)}{2}\left(\log\left(2\pi\right)+\log\left(\sigma^{2}\right)\right)-\frac{1}{\sigma^{2}}\sum_{i}\epsilon_{i}^{2} If it is fit by MLE then the (exact) unconditional maximum likelihood is returned. .. math:: -\frac{n}{2}log\left(2\pi\right)-\frac{n}{2}\log\left(\sigma^{2}\right)+\frac{1}{2}\left|V_{p}^{-1}\right|-\frac{1}{2\sigma^{2}}\left(y_{p}-\mu_{p}\right)^{\prime}V_{p}^{-1}\left(y_{p}-\mu_{p}\right)-\frac{1}{2\sigma^{2}}\sum_{t=p+1}^{n}\epsilon_{i}^{2} where :math:`\mu_{p}` is a (`p` x 1) vector with each element equal to the mean of the AR process and :math:`\sigma^{2}V_{p}` is the (`p` x `p`) variance-covariance matrix of the first `p` observations. r"N)r&rjrp)r<r,rrr ro:s!  z AR.loglikecCs|j}t||ddS)a Return the gradient of the loglikelihood at params. Parameters ---------- params : array-like The parameter values at which to evaluate the score function. Notes ----- Returns numerical gradient. g:Œ0âŽyE>)Úepsilon)ror)r<r,rorrr Úscoreas zAR.scorecCsdS)z% Not Implemented Yet Nr)r<r,rrr Ú informationqszAR.informationcCs|j}t||ƒS)z4 Returns numerical hessian for now. )ror)r<r,rorrr Úhessianwsz AR.hessiancCs:|j}t||dd}t |¡}|r0t|d|d}||_|S)zt Private method to build the RHS matrix for estimation. Columns are trend terms then lags. Zboth)ÚmaxlagZtrimT)ZprependÚtrend)r0rrZget_trendorderrr.)r<r%rvr0rYr.rrr Ú_stackX~s z AR._stackXrnrRc Cs|j}||d…}||_| ||¡}||_|j}td|ƒ}i} |dkr´xPt||dƒD]>} ||| d…} t| ƒj| |d|ddd} t d|ƒ| | <qVWt dd „t | ƒDƒƒ\} }nfd }x`t||dd ƒD]L} ||| d…} t| ƒj| |d|d d d} d}t   | jd ¡|krÊ| }PqÊW|S) a2 Select the lag order according to the information criterion. Parameters ---------- maxlag : int The highest lag length tried. See `AR.fit`. ic : str {'aic','bic','hqic','t-stat'} Criterion used for selecting the optimal lag length. See `AR.fit`. trend : str {'c','nc'} Whether to include a constant or not. 'c' - include constant. 'nc' - no constant. Returns ------- bestlag : int Best lag according to IC. Nr7zt-statréd)rur&Ú full_outputrvÚmaxiterÚdispzfit.css|]\}}||fVqdS)Nr)Ú.0rBÚresrrr ú ³sz"AR.select_order..g»jg RQú?r)é#)r0rerwrYr.rZrrÚfitÚevalrXrrÚabsÚtvalues)r<ruÚicrvr&r0rerYrBZresultsZlagZ endog_tmpr€ZbesticZbestlagÚstoprrr Ú select_orderŒs6       zAR.select_orderr"TÚlbfgsrr7c  sJ| ¡}|dkrtd|ƒ‚||_||_||_t|jƒ} |j}|dkr^ttd| ddƒƒ}|}|dk r–| ¡}|dkr†td|ƒ‚|  ||||¡}||_ ||d…dd…f}|  ||¡}|j }t  |j||¡|_||_||_|d krt||ƒ ¡}|j}| ||_|j|j|_nä|d krü| ¡}| |_|dkrLt||ƒ ¡j}n*t|ƒ||krvtd t|ƒ||fƒ‚| |¡}|d krº|  d d¡|  dd¡|  dd¡|  dd¡tt|ƒjf|||| | | dœ| —Ž}|j}|jrü| |¡}d|_tj  |¡}t !||j"¡}t#|||ƒ}|d krB| rB|j$|_$|j%|_%t&|ƒS)a Fit the unconditional maximum likelihood of an AR(p) process. Parameters ---------- maxlag : int If `ic` is None, then maxlag is the lag length used in fit. If `ic` is specified then maxlag is the highest lag order used to select the correct lag order. If maxlag is None, the default is round(12*(nobs/100.)**(1/4.)) method : str {'cmle', 'mle'}, optional cmle - Conditional maximum likelihood using OLS mle - Unconditional (exact) maximum likelihood. See `solver` and the Notes. ic : str {'aic','bic','hic','t-stat'} Criterion used for selecting the optimal lag length. aic - Akaike Information Criterion bic - Bayes Information Criterion t-stat - Based on last lag hqic - Hannan-Quinn Information Criterion If any of the information criteria are selected, the lag length which results in the lowest value is selected. If t-stat, the model starts with maxlag and drops a lag until the highest lag has a t-stat that is significant at the 95 % level. trend : str {'c','nc'} Whether to include a constant or not. 'c' - include constant. 'nc' - no constant. The below can be specified if method is 'mle' transparams : bool, optional Whether or not to transform the parameters to ensure stationarity. Uses the transformation suggested in Jones (1980). start_params : array-like, optional A first guess on the parameters. Default is cmle estimates. solver : str or None, optional Solver to be used if method is 'mle'. The default is 'lbfgs' (limited memory Broyden-Fletcher-Goldfarb-Shanno). Other choices are 'bfgs', 'newton' (Newton-Raphson), 'nm' (Nelder-Mead), 'cg' - (conjugate gradient), 'ncg' (non-conjugate gradient), and 'powell'. maxiter : int, optional The maximum number of function evaluations. Default is 35. tol : float The convergence tolerance. Default is 1e-08. full_output : bool, optional If True, all output from solver will be available in the Results object's mle_retvals attribute. Output is dependent on the solver. See Notes for more information. disp : bool, optional If True, convergence information is output. callback : function, optional Called after each iteration as callback(xk) where xk is the current parameter vector. kwargs See Notes for keyword arguments that can be passed to fit. References ---------- Jones, R.H. 1980 "Maximum likelihood fitting of ARMA models to time series with missing observations." `Technometrics`. 22.3. 389-95. See also -------- statsmodels.base.model.LikelihoodModel.fit )r"ZywrRzMethod %s not recognizedNé gY@gÐ?)ÚaicÚbicÚhqiczt-statzic option %s not understoodr"rRz6Length of start params is %d. There are %d parameters.r‡Zpgtolg:Œ0âŽyE>ZfactrÚmZ approx_gradT)rEr&rzryr{ÚcallbackF)'Úlowerr#r&rvrkr[r0ÚintÚroundr†r%rwr.rZmake_lag_namesZ endog_namesZ exog_namesrerYr r€r,rdrhrirFÚ setdefaultr8rrDrÚlinalgZpinvrrIÚ ARResultsZ mle_retvalsZ mle_settingsÚARResultsWrapper)r<rur&r„rvrkrEÚsolverrzryr{rÚkwargsrdr0r%rerYr.Zarfitr,ZmlefitZ pinv_exogÚnormalized_cov_params)r?rr r€ÄsrF                 zAR.fit)NNr5)N)NNF)rnrR) Nr"NrnTNr‡rr7r7N)Ú__name__Ú __module__Ú __qualname__ÚtsbaseZ_tsa_docÚbaseZ_missing_param_docÚ__doc__r9r@rDrFrQrTr_rcrjrprorrrsrtrwr†r€Ú __classcell__rr)r?r r8s.   # T %' 8cseZdZdZiZd$‡fdd„ Zeƒdd„ƒZeƒdd „ƒZe d d „ƒZ e d d „ƒZ e dd„ƒZ e dd„ƒZ e dd„ƒZe dd„ƒZe dd„ƒZe dd„ƒZe dd„ƒZd%dd„Zejj d¡Zd  d¡Zd edd!…ed"d#…eed#d…¡e_‡ZS)&r“u¹ Class to hold results from fitting an AR model. Parameters ---------- model : AR Model instance Reference to the model that is fit. params : array The fitted parameters from the AR Model. normalized_cov_params : array inv(dot(X.T,X)) where X is the lagged values. scale : float, optional An estimate of the scale of the model. Returns ------- **Attributes** aic : float Akaike Information Criterion using Lutkephol's definition. :math:`log(sigma) + 2*(1 + k_ar + k_trend)/nobs` bic : float Bayes Information Criterion :math:`\log(\sigma) + (1 + k_ar + k_trend)*\log(nobs)/nobs` bse : array The standard errors of the estimated parameters. If `method` is 'cmle', then the standard errors that are returned are the OLS standard errors of the coefficients. If the `method` is 'mle' then they are computed using the numerical Hessian. fittedvalues : array The in-sample predicted values of the fitted AR model. The `k_ar` initial values are computed via the Kalman Filter if the model is fit by `mle`. fpe : float Final prediction error using Lütkepohl's definition ((n_totobs+k_trend)/(n_totobs-k_ar-k_trend))*sigma hqic : float Hannan-Quinn Information Criterion. k_ar : float Lag length. Sometimes used as `p` in the docs. k_trend : float The number of trend terms included. 'nc'=0, 'c'=1. llf : float The loglikelihood of the model evaluated at `params`. See `AR.loglike` model : AR model instance A reference to the fitted AR model. nobs : float The number of available observations `nobs` - `k_ar` n_totobs : float The number of total observations in `endog`. Sometimes `n` in the docs. params : array The fitted parameters of the model. pvalues : array The p values associated with the standard errors. resid : array The residuals of the model. If the model is fit by 'mle' then the pre-sample residuals are calculated using fittedvalues from the Kalman Filter. roots : array The roots of the AR process are the solution to (1 - arparams[0]*z - arparams[1]*z**2 -...- arparams[p-1]*z**k_ar) = 0 Stability requires that the roots in modulus lie outside the unit circle. scale : float Same as sigma2 sigma2 : float The variance of the innovations (residuals). trendorder : int The polynomial order of the trend. 'nc' = None, 'c' or 't' = 0, 'ct' = 1, etc. tvalues : array The t-values associated with `params`. Nçð?c sštt|ƒ ||||¡tƒ|_|j|_t|jƒ}||_|j |_ |j |_ |j }||_ |j }||_ d}|dkrr|d}||_ |||_||j|_|j_dS)Nrr7)r8r“r9rÚ_cacherdr[r0Ún_totobsrYrer%r.Ú trendorderÚdf_modelZdf_residr4) r<r4r,r—Úscaler¡r%r.r¢)r?rr r9¨s$  zARResults.__init__cCs0|j}|jdkr$d|jt|jƒS|jjSdS)Nr"gð?)r4r&rdr!Úresidri)r<r4rrr ri½s zARResults.sigma2cCs|jS)N)ri)r<rrr r¤ÅszARResults.scalecCsz|jjdkrL|j}t ||¡}||j|j|j}t t  |j |d¡¡St |j |jj ƒ}t t  tj |¡ ¡¡SdS)Nr")r¤)r4r&r¥rrrdr%r.ZsqrtraZ cov_paramsrr,ror’r )r<r¥rhZ ols_scaleZhessrrr ÚbseÉs  z ARResults.bsecCst t |j¡¡dS)Nr)r Zsfrr‚rƒ)r<rrr ÚpvaluesÔszARResults.pvaluescCs t |j¡dd|j|jS)Nrr7)rrfrir£rd)r<rrr r‰Øsz ARResults.aiccCs4|j}t |j¡dt t |¡¡|d|jS)Nrr7)rdrrfrir£)r<rdrrr r‹ãs zARResults.hqiccCs"|j}|j}|||||jS)N)rdr£ri)r<rdr£rrr Úfpeðsz ARResults.fpecCs*|j}t |j¡d|jt |¡|S)Nr7)rdrrfrir£)r<rdrrr rŠ÷sz ARResults.biccCsB|j}|j ¡}|jdkr.||jd…|jS|j ¡|jSdS)Nr")r4r0rWr&r%r^)r<r4r0rrr r¥s   zARResults.residcCs*|j}t tjd|j|d… f¡dS)Nr7r))r.rÚrootsr\r,)r<rBrrr r©szARResults.rootscCs|j |j¡S)N)r4r_r,)r<rrr r^szARResults.fittedvaluesFcCs|j}|j ||||¡}|S)N)r,r4r_)r<r$rLr'r,rMrrr r_szARResults.predictÚ a confint : bool, float Whether to return confidence intervals. If `confint` == True, 95 % confidence intervals are returned. Else if `confint` is a float, then it is assumed to be the alpha value of the confidence interval. That is confint == .05 returns a 95% confidence interval, and .10 would return a 90% confidence interval.ééé)NrŸ)NNF)r˜r™ršrr r9rrir¤rr¦r§r‰r‹r¨rŠr¥r©r^r_rÚsplitZpreddocZ extra_docÚjoinržrr)r?r r“[s(I      r“c@s4eZdZiZe ejje¡ZiZ e ejj e ¡Z dS)r”N) r˜r™ršZ_attrsÚwrapZ union_dictsr›ZTimeSeriesResultsWrapperZ _wrap_attrsZ_methodsZ _wrap_methodsrrrr r”6s   r”Ú__main__é )rurRZbfgsiôg»½×Ùß|Û=)rur&r•rzZgtoli¤ÚA)Zyearr>)Z start_dateÚlengthzA-DEC)r$Zperiodsr>)r>)rU)r&ruiÌiÉiÄiËiÎiÏißiíg ~@gà?)r)r)`Z __future__rZstatsmodels.compat.pythonrrrrrZnumpyrrr Z numpy.linalgr r Z scipy.statsr Z#statsmodels.regression.linear_modelr Zstatsmodels.tsa.tsatoolsrrrrZstatsmodels.tsa.base.tsa_modelZtsarœZ tsa_modelr›Zstatsmodels.base.modelr4Zstatsmodels.tools.decoratorsrrrZstatsmodels.tools.numdiffrrZ$statsmodels.tsa.kalmanf.kalmanfilterrZstatsmodels.base.wrapperÚwrapperr°Zstatsmodels.tsa.vector_arrÚ__all__r!r(r2ZTimeSeriesModelrZTimeSeriesModelResultsr“ZResultsWrapperr”Zpopulate_wrapperr˜Zstatsmodels.apiZapiZsmZdatasetsZsunspotsÚloadr0Zar_olsr€Zres_olsZar_mleZ res_mle_bfgsZpandasZscikits.timeseriesZ timeseriesZtsZDateZd1Z date_arrayr[Zts_drZ DatetimeIndexZdatetimeZ pandas_drZaranger=rlZ fromordinalZ toordinalZastyperZdt_datesZSeriesÚmodr}ZIBMZdiffÚwZthetarrrr Úsj       '\         $