p7]c@s(dZddlmZddlmZmZmZddlZddl m Z ddl m Z ddl m Z ddlmZmZdd lmZmZddljjZddljjZdd lmZddlZdd lmZdd lm Z d dl!m"Z#dZ$dddddgZ%dZ&dZ'dej(fdYZ)de)fdYZ*de)fdYZ+de+fdYZ,de*fdYZ-d de.e/e0dZ1d e0dZ2d ej3fd!YZ4d"e4fd#YZ5d$ej6fd%YZ7ej8e7e4dS(&sH This module implements standard regression models: Generalized Least Squares (GLS) Ordinary Least Squares (OLS) Weighted Least Squares (WLS) Generalized Least Squares with autoregressive error terms GLSAR(p) Models are specified with an endogenous response variable and an exogenous design matrix and are fit using their `fit` method. Subclasses that have more complicated covariance matrices should write over the 'whiten' method as the fit method prewhitens the response by calling 'whiten'. General reference for regression models: D. C. Montgomery and E.A. Peck. "Introduction to Linear Regression Analysis." 2nd. Ed., Wiley, 1992. Econometrics references for regression models: R. Davidson and J.G. MacKinnon. "Econometric Theory and Methods," Oxford, 2004. W. Green. "Econometric Analysis," 5th ed., Pearson, 2003. i(tprint_function(tlrangetlziptrangeN(ttoeplitz(tstats(toptimize(t chain_dott pinv_extended(tcache_readonlytcache_writable(t _ELRegOpts(tInvalidTestWarning(tPredictionResultsi(t _predictionsrestructuredtext entGLStWLStOLStGLSARR sI Return a regularized fit to a linear regression model. Parameters ---------- method : string 'elastic_net' and 'sqrt_lasso' are currently implemented. alpha : scalar or array-like The penalty weight. If a scalar, the same penalty weight applies to all variables in the model. If a vector, it must have the same length as `params`, and contains a penalty weight for each coefficient. L1_wt: scalar The fraction of the penalty given to the L1 penalty term. Must be between 0 and 1 (inclusive). If 0, the fit is a ridge fit, if 1 it is a lasso fit. start_params : array-like Starting values for ``params``. profile_scale : bool If True the penalized fit is computed using the profile (concentrated) log-likelihood for the Gaussian model. Otherwise the fit uses the residual sum of squares. refit : bool If True, the model is refit using only the variables that have non-zero coefficients in the regularized fit. The refitted model is not regularized. distributed : bool If True, the model uses distributed methods for fitting, will raise an error if True and partitions is None. generator : function generator used to partition the model, allows for handling of out of memory/parallel computing. partitions : scalar The number of partitions desired for the distributed estimation. threshold : scalar or array-like The threshold below which coefficients are zeroed out, only used for distributed estimation Returns ------- A RegularizedResults instance. Notes ----- The elastic net uses a combination of L1 and L2 penalties. The implementation closely follows the glmnet package in R. The function that is minimized is: .. math:: 0.5*RSS/n + alpha*((1-L1\_wt)*|params|_2^2/2 + L1\_wt*|params|_1) where RSS is the usual regression sum of squares, n is the sample size, and :math:`|*|_1` and :math:`|*|_2` are the L1 and L2 norms. For WLS and GLS, the RSS is calculated using the whitened endog and exog data. Post-estimation results are based on the same data used to select variables, hence may be subject to overfitting biases. The elastic_net method uses the following keyword arguments: maxiter : int Maximum number of iterations cnvrg_tol : float Convergence threshold for line searches zero_tol : float Coefficients below this threshold are treated as zero. The square root lasso approach is a variation of the Lasso that is largely self-tuning (the optimal tuning parameter does not depend on the standard deviation of the regression errors). If the errors are Gaussian, the tuning parameter can be taken to be alpha = 1.1 * np.sqrt(n) * norm.ppf(1 - 0.05 / (2 * p)) where n is the sample size and p is the number of predictors. The square root lasso uses the following keyword arguments: zero_tol : float Coefficients below this threshold are treated as zero. References ---------- Friedman, Hastie, Tibshirani (2008). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33(1), 1-22 Feb 2010. A Belloni, V Chernozhukov, L Wang (2011). Square-root Lasso: pivotal recovery of sparse signals via conic programming. Biometrika 98(4), 791-806. https://arxiv.org/pdf/1009.5689.pdf cCs|dkrdStj|j}|jdkrItj||}n|jdkr|j|fkrtd|||fndtj|}nR|j||fkrtd|||fntj j tj j |j }||fS(s Returns sigma (matrix, nobs by nobs) for GLS and the inverse of its Cholesky decomposition. Handles dimensions and checks integrity. If sigma is None, returns None, None. Otherwise returns sigma, cholsigmainv. iisFSigma must be a scalar, 1d of length %s or a 2d array of shape %s x %sN(NN( tNonetnptasarraytsqueezetndimtrepeattshapet ValueErrortsqrttlinalgtcholeskytinvtT(tsigmatnobst cholsigmainv((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt _get_sigmas !tRegressionModelcBseZdZdZdZedZejdZedZejdZdZ dd d d d Z d d Z d d d Z RS(sp Base class for linear regression models. Should not be directly called. Intended for subclassing. cKs<tt|j||||jjddddgdS(Nt pinv_wexogtwendogtwexogtweights(tsuperR$t__init__t _data_attrtextend(tselftendogtexogtkwargs((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR*scCsb|j|j|_|j|j|_t|jjd|_d|_ d|_ d|_ dS(Ni( twhitenR/R'R.R&tfloatRR!Rt _df_modelt _df_residtrank(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt initializes   cCs\|jdkrU|jdkr9tjj|j|_nt|j|j|_n|jS(s The model degree of freedom, defined as the rank of the regressor matrix minus 1 if a constant is included. N( R3RR5RRt matrix_rankR/R2t k_constant(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytdf_models cCs ||_dS(N(R3(R-tvalue((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR9scCsV|jdkrO|jdkr9tjj|j|_n|j|j|_n|jS(s The residual degree of freedom, defined as the number of observations minus the rank of the regressor matrix. N(R4RR5RRR7R/R!(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytdf_resids cCs ||_dS(N(R4(R-R:((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR;scCstddS(NsSubclasses should implement.(tNotImplementedError(R-tX((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR1stpinvt nonrobustc Ks|dkrt|do6t|do6t|dst|j\|_}tj|jtj|j|_||_tj j tj ||_ ntj|j|j }n#|dkrt|dot|dot|dot|dstj j|j\}} || |_|_tj jtj| j| |_tj j| dd|_tj j | |_ n|j|j}} tj|j|j |_} tj j| | }n td |jd krt|j |j|_n|jd kr-|j|j |_nt|trit ||d|jd |d |d |} n-t!||d|jd |d |d ||} t"| S(s Full fit of the model. The results include an estimate of covariance matrix, (whitened) residuals and an estimate of scale. Parameters ---------- method : str, optional Can be "pinv", "qr". "pinv" uses the Moore-Penrose pseudoinverse to solve the least squares problem. "qr" uses the QR factorization. cov_type : str, optional See `regression.linear_model.RegressionResults` for a description of the available covariance estimators cov_kwds : list or None, optional See `linear_model.RegressionResults.get_robustcov_results` for a description required keywords for alternative covariance estimators use_t : bool, optional Flag indicating to use the Student's t distribution when computing p-values. Default behavior depends on cov_type. See `linear_model.RegressionResults.get_robustcov_results` for implementation details. Returns ------- A RegressionResults class instance. See Also -------- regression.linear_model.RegressionResults regression.linear_model.RegressionResults.get_robustcov_results Notes ----- The fit method uses the pseudoinverse of the design/exogenous variables to solve the least squares minimization. R>R%tnormalized_cov_paramsR5tqrtexog_Qtexog_Rismethod has to be "pinv" or "qr"tcov_typetcov_kwdstuse_tN(#thasattrRR'R%Rtdott transposeR@twexog_singular_valuesRR7tdiagR5R&RARBRCRRtsvdteffectstsolveRR3RR2R8R4R!R;t isinstanceRt OLSResultstRegressionResultstRegressionResultsWrapper( R-tmethodRDRERFR0tsingular_valuestbetatQtRRMtlfit((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytfitsN(  ! $      cCs(|dkr|j}ntj||S(s Return linear predicted values from a design matrix. Parameters ---------- params : array-like Parameters of a linear model exog : array-like, optional. Design / exogenous data. Model exog is used if None. Returns ------- An array of fitted values Notes ----- If the model has not yet been fit, params is not optional. N(RR/RRH(R-tparamsR/((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytpredictSs  cCsY|j||}|dkr7ddlm}|}n|d|dtj|}|S(s Returns a random number generator for the predictive distribution. Parameters ---------- params : array-like The model parameters (regression coefficients). scale : scalar The variance parameter. exog : array-like The predictor variable matrix. dist_class : class A random number generator class. Must take 'loc' and 'scale' as arguments and return a random number generator implementing an ``rvs`` method for simulating random values. Defaults to Gaussian. Returns ------- gen Frozen random number generator object with mean and variance determined by the fitted linear model. Use the ``rvs`` method to generate random values. Notes ----- Due to the behavior of ``scipy.stats.distributions objects``, the returned random number generator must be called with ``gen.rvs(n)`` where ``n`` is the number of observations in the data set used to fit the model. If any other value is used for ``n``, misleading results will be produced. i(tnormtloctscaleN(R[Rtscipy.stats.distributionsR\RR(R-RZR^R/t dist_classRYR\tgen((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytget_distributionns   N(t__name__t __module__t__doc__R*R6tpropertyR9tsetterR;R1RRYR[Rb(((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR$s       ] cBseZdiejd6ejejd6Zd dd dZdZ dZ d e dZ dd d d e e d Zee_RS( s8 Generalized least squares model with a general covariance structure. %(params)s sigma : scalar or array `sigma` is the weighting matrix of the covariance. The default is None for no scaling. If `sigma` is a scalar, it is assumed that `sigma` is an n x n diagonal matrix with the given scalar, `sigma` as the value of each diagonal element. If `sigma` is an n-length vector, then `sigma` is assumed to be a diagonal matrix with the given `sigma` on the diagonal. This should be the same as WLS. %(extra_params)s Attributes ---------- pinv_wexog : array `pinv_wexog` is the p x n Moore-Penrose pseudoinverse of `wexog`. cholsimgainv : array The transpose of the Cholesky decomposition of the pseudoinverse. df_model : float p - 1, where p is the number of regressors including the intercept. of freedom. df_resid : float Number of observations n less the number of parameters p. llf : float The value of the likelihood function of the fitted model. nobs : float The number of observations n. normalized_cov_params : array p x p array :math:`(X^{T}\Sigma^{-1}X)^{-1}` results : RegressionResults instance A property that returns the RegressionResults class if fit. sigma : array `sigma` is the n x n covariance structure of the error terms. wexog : array Design matrix whitened by `cholsigmainv` wendog : array Response variable whitened by `cholsigmainv` Notes ----- If sigma is a function of the data making one of the regressors a constant, then the current postestimation statistics will not be correct. Examples -------- >>> import numpy as np >>> import statsmodels.api as sm >>> data = sm.datasets.longley.load(as_pandas=False) >>> data.exog = sm.add_constant(data.exog) >>> ols_resid = sm.OLS(data.endog, data.exog).fit().resid >>> res_fit = sm.OLS(ols_resid[1:], ols_resid[:-1]).fit() >>> rho = res_fit.params `rho` is a consistent estimator of the correlation of the residuals from an OLS fit of the longley data. It is assumed that this is the true rho of the AR process data. >>> from scipy.linalg import toeplitz >>> order = toeplitz(np.arange(16)) >>> sigma = rho**order `sigma` is an n x n matrix of the autocorrelation structure of the data. >>> gls_model = sm.GLS(data.endog, data.exog, sigma=sigma) >>> gls_results = gls_model.fit() >>> print(gls_results.summary()) RZt extra_paramstnonec Ksit|t|\}}tt|j||d|d|d|d|||jjddgdS(NtmissingthasconstR R"(R#tlenR)RR*R+R,(R-R.R/R RjRkR0R"((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR*s   cCstj|}|jdks0|jjdkr4|S|jjdkr~|jdkr`||jS||jdddfSntj|j|SdS(s GLS whiten method. Parameters ---------- X : array-like Data to be whitened. Returns ------- np.dot(cholsigmainv,X) See Also -------- regression.GLS iN((RRR RRRR"RH(R-R=((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR1s! cCs|jd}tj|jtj|j|ddd}tj| |}|dtjtj||8}tj|j r|j j dkrtj j |j }|d|d8}q|dtjtj|j 8}n|S(s) Returns the value of the Gaussian log-likelihood function at params. Given the whitened design matrix, the log-likelihood is evaluated at the parameter vector `params` for the dependent variable `endog`. Parameters ---------- params : array-like The parameter estimates Returns ------- loglike : float The value of the log-likelihood function for a GLS Model. Notes ----- The log-likelihood function for the normal distribution is .. math:: -\frac{n}{2}\log\left(\left(Y-\hat{Y}\right)^{\prime}\left(Y-\hat{Y}\right)\right)-\frac{n}{2}\left(1+\log\left(\frac{2\pi}{n}\right)\right)-\frac{1}{2}\log\left(\left|\Sigma\right|\right) Y and Y-hat are whitened. g@itaxisiig?( R!RtsumR&RHR'tlogtpitanyR RRtslogdet(R-RZtnobs2tSSRtllftdet((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytloglike s /"&cCse|jdks!|jjdkr8tj|jjdS|jjdkrQ|jStj|jSdS(sWeights for calculating Hessian Parameters ---------- params : ndarray parameter at which Hessian is evaluated scale : None or float If scale is None, then the default scale will be calculated. Default scale is defined by `self.scaletype` and set in fit. If scale is not None, then it is used as a fixed scale. observed : bool If True, then the observed Hessian is returned. If false then the expected information matrix is returned. Returns ------- hessian_factor : ndarray, 1d A 1d weight vector used in the calculation of the Hessian. The hessian is obtained by `(exog.T * hessian_factor).dot(exog)` iiN(( R RRRtonesR/RR"RK(R-RZR^tobserved((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pythessian_factor4s !t elastic_netgg?c Ks|jdk rB|tjdtj|jt|j}nt|j|j j d|d|d|d|d|d||}dd l m } m } | ||j} | | S( NiRStalphatL1_wtt start_paramst profile_scaletrefiti(tRegularizedResultstRegularizedResultsWrapper(R RRRnRKRlR.RR&R'tfit_regularizedtstatsmodels.base.elastic_netRRRZ( R-RSR|R}R~RRR0trsltRRtrrslt((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRQs3  N(RcRdtbaset_model_params_doct_missing_param_doct_extra_param_docReRR*R1RwtTrueRztFalseRt_fit_regularized_doc(((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRsH  *  cBseZdiejd6ejejd6Zddd dZdZ dZ d e dZ d d dd e e d Zee_RS( sJ A regression model with diagonal but non-identity covariance structure. The weights are presumed to be (proportional to) the inverse of the variance of the observations. That is, if the variables are to be transformed by 1/sqrt(W) you must supply weights = 1/W. %(params)s weights : array-like, optional 1d array of weights. If you supply 1/W then the variables are pre- multiplied by 1/sqrt(W). If no weights are supplied the default value is 1 and WLS results are the same as OLS. %(extra_params)s Attributes ---------- weights : array The stored weights supplied as an argument. See Also -------- regression.GLS Examples -------- >>> import numpy as np >>> import statsmodels.api as sm >>> Y = [1,3,4,5,2,3,4] >>> X = range(1,8) >>> X = sm.add_constant(X) >>> wls_model = sm.WLS(Y,X, weights=list(range(1,8))) >>> results = wls_model.fit() >>> results.params array([ 2.91666667, 0.0952381 ]) >>> results.tvalues array([ 2.0652652 , 0.35684428]) >>> print(results.t_test([1, 0])) >>> print(results.f_test([0, 1])) Notes ----- If the weights are a function of the data, then the post estimation statistics such as fvalue and mse_model might not be correct, as the package does not yet support no-constant regression. RZRhg?Ric KsLtj|}|jd kr|dkred|kre|ddk retj|t|d}qtj|t|}nt|dkrtj|jg}n |j}tt|j ||d|d|d|||j jd}|j }|tj ||}|j |krH|jd|krHtdndS( Ntdropt missing_idxiRjR(Rkis/Weights must be scalar or same length as design((RtarrayRRRRlRR)RR*R/R(RntsizeR(R-R.R/R(RjRkR0R!((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR*s   "cCsitj|}|jdkr2|tj|jS|jdkretj|jdddf|SdS(s Whitener for WLS model, multiplies each column by sqrt(self.weights) Parameters ---------- X : array-like Data to be whitened Returns ------- whitened : array-like sqrt(weights)*X iiN(RRRRR(R(R-R=((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR1s cCs|jd}tj|jtj|j|ddd}tj| |}|dtjtj||8}|dtjtj|j7}|S(s Returns the value of the gaussian log-likelihood function at params. Given the whitened design matrix, the log-likelihood is evaluated at the parameter vector `params` for the dependent variable `Y`. Parameters ---------- params : array-like The parameter estimates. Returns ------- llf : float The value of the log-likelihood function for a WLS Model. Notes -------- .. math:: -\frac{n}{2}\log\left(Y-\hat{Y}\right)-\frac{n}{2}\left(1+\log\left(\frac{2\pi}{n}\right)\right)-\frac{1}{2}log\left(\left|W\right|\right) where :math:`W` is a diagonal matrix g@iRmiig?( R!RRnR&RHR'RoRpR((R-RZRsRtRu((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRws  /"#cCs|jS(sWeights for calculating Hessian Parameters ---------- params : ndarray parameter at which Hessian is evaluated scale : None or float If scale is None, then the default scale will be calculated. Default scale is defined by `self.scaletype` and set in fit. If scale is not None, then it is used as a fixed scale. observed : bool If True, then the observed Hessian is returned. If false then the expected information matrix is returned. Returns ------- hessian_factor : ndarray, 1d A 1d weight vector used in the calculation of the Hessian. The hessian is obtained by `(exog.T * hessian_factor).dot(exog)` (R((R-RZR^Ry((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRzsR{gc Ks|tj|jt|j}t|j|jjd|d|d|d|d|d||}ddlm } m } | ||j } | | S( NRSR|R}R~RRi(RR( RRnR(RlRR&R'RRRRRZ( R-RSR|R}R~RRR0RRRR((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRs#  N(RcRdRRRRReRR*R1RwRRzRRR(((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRjs/    cBseZdiejd6ejejd6ZddddZddZ dZ ddZ dZ dd Z ded Zd d d deedZee_dZdZRS(s A simple ordinary least squares model. %(params)s %(extra_params)s Attributes ---------- weights : scalar Has an attribute weights = array(1.0) due to inheritance from WLS. See Also -------- GLS Examples -------- >>> import numpy as np >>> >>> import statsmodels.api as sm >>> >>> Y = [1,3,4,5,2,3,4] >>> X = range(1,8) >>> X = sm.add_constant(X) >>> >>> model = sm.OLS(Y,X) >>> results = model.fit() >>> results.params array([ 2.14285714, 0.25 ]) >>> results.tvalues array([ 1.87867287, 0.98019606]) >>> print(results.t_test([1, 0])) >>> print(results.f_test(np.identity(2))) Notes ----- No constant is added by the model unless you are using formulas. RZRhRicKsNtt|j||d|d||d|jkrJ|jjdndS(NRjRkR((R)RR*t _init_keystremove(R-R.R/RjRkR0((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR*Cs cCs|jd}t|j}|jtj|j|}t|drW||j8}ntj|d}|dkr| tj dtj |tj |||}n+| tj dtj ||d|}|S(s The likelihood function for the OLS model. Parameters ---------- params : array-like The coefficients with which to estimate the log-likelihood. scale : float or None If None, return the profile (concentrated) log likelihood (profiled over the scale parameter), else return the log-likelihood using the given scale value. Returns ------- The likelihood function evaluated at params. g@toffsetiN( R!R2R.RRHR/RGRRnRRoRp(R-RZR^RsR!tresidtssrRu((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRwJs  7+cCs|S(s= OLS model whitener does nothing: returns Y. ((R-tY((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR1iscCst|ds|jntj|j|}|j |}|dkr|jdtj|jj|}|tj||7}|j ||S| |SdS(s Evaluate the score function at a given point. The score corresponds to the profile (concentrated) log-likelihood in which the scale parameter has been profiled out. Parameters ---------- params : array-like The parameter vector at which the score function is computed. scale : float or None If None, return the profile (concentrated) log likelihood (profiled over the scale parameter), else return the log-likelihood using the given scale value. Returns ------- The score vector. t _wexog_xprodiN( RGt_setup_score_hessRRHRt_wexog_x_wendogRt _wendog_xprodRR!(R-RZR^txtxbtsdrR((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytscoreos   cCs{|j}t|dr(||j}ntj|||_tj|jj|j|_ tj|jj||_ dS(NR( R&RGRRRnRRHR'RRR(R-ty((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRs  cCst|ds|jntj|j|}|dkr|jdtj|jj|}|tj||7}d|jd|}|j|tj |||d}|j |dS|j |SdS(s Evaluate the Hessian function at a given point. Parameters ---------- params : array-like The parameter vector at which the Hessian is computed. scale : float or None If None, return the profile (concentrated) log likelihood (profiled over the scale parameter), else return the log-likelihood using the given scale value. Returns ------- The Hessian matrix. RiiN( RGRRRHRRRRRtouterR!(R-RZR^RRtssrpthm((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pythessians   %cCstj|jjdS(sWeights for calculating Hessian Parameters ---------- params : ndarray parameter at which Hessian is evaluated scale : None or float If scale is None, then the default scale will be calculated. Default scale is defined by `self.scaletype` and set in fit. If scale is not None, then it is used as a fixed scale. observed : bool If True, then the observed Hessian is returned. If false then the expected information matrix is returned. Returns ------- hessian_factor : ndarray, 1d A 1d weight vector used in the calculation of the Hessian. The hessian is obtained by `(exog.T * hessian_factor).dot(exog)` i(RRxR/R(R-RZR^Ry((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRzsR{gg?cKsO|dkr%d|}t|nidd6dd6dd 6} | j||dkrd d lm} m} |j||| d } | || } | | Sd d lm}|d kr|j|S|ri}i}i}n'idd6}idd6}idd6}||d|d|d|d|d|d|d|d|dt|  S(NR{t sqrt_lassos'Unknown method '%s' for fit_regularizedi2tmaxiterg|=t cnvrg_tolg:0yE>tzero_toli(RR(tfit_elasticnetiiR^RSR|R}R~t loglike_kwdst score_kwdst hess_kwdsRt check_step(R{R( RtupdateRRRt _sqrt_lassoRt _fit_ridgeR(R-RSR|R}R~RRR0tmsgtdefaultsRRRZtresultsRRRR((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRs>            c Csyddl}Wn#tk r5d}t|nXt|j}|jjd}|jdd|ddf}|jd|ddf} |j|j|df| dddf<|jgggd|dd|df} x/t dd|dD]} d| | | f>> import statsmodels.api as sm >>> X = range(1,8) >>> X = sm.add_constant(X) >>> Y = [1,3,4,5,8,10,9] >>> model = sm.GLSAR(Y, X, rho=2) >>> for i in range(6): ... results = model.fit() ... print("AR coefficients: {0}".format(model.rho)) ... rho, sigma = sm.regression.yule_walker(results.resid, ... order=model.order) ... model = sm.GLSAR(Y, X, rho) ... AR coefficients: [ 0. 0.] AR coefficients: [-0.52571491 -0.84496178] AR coefficients: [-0.6104153 -0.86656458] AR coefficients: [-0.60439494 -0.857867 ] AR coefficients: [-0.6048218 -0.85846157] AR coefficients: [-0.60479146 -0.85841922] >>> results.params array([-0.66661205, 1.60850853]) >>> results.tvalues array([ -2.10304127, 21.8047269 ]) >>> print(results.t_test([1, 0])) >>> print(results.f_test(np.identity(2))) Or, equivalently >>> model2 = sm.GLSAR(Y, X, rho=2) >>> res = model2.iterative_fit(maxiter=6) >>> model2.rho array([-0.60479146, -0.85841922]) Notes ----- GLSAR is considered to be experimental. The linear autoregressive process of order p--AR(p)--is defined as: TODO RZRhiRicKst|tjr9||_tj|jtj|_nvtjtj||_t |jj dkr{t dn|jj dkrd|j_ n|jj d|_|dkrt t|j|tj|j ddfd||n"t t|j||d||dS(Niis*AR parameters must be a scalar or a vectorRj(ii((i(RORtinttordertzerostfloat64trhoRRRlRRRR)RR*Rx(R-R.R/RRjR0((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR*s  . ig-C6?c Kst}d}igd6|jgd6}xt|dD]}t|drU|`n|j|j}|dj|j|dkr|j}nKt j t j ||jt j |} | |krt }Pn|j}t |jd|jdd \|_} |dj|jq7W| rY|dkrYt|drL|`n|jn|jd ||}|d|_|s|jdj|j|jd7_n||_|S( s  Perform an iterative two-stage procedure to estimate a GLS model. The model is assumed to have AR(p) errors, AR(p) parameters and regression coefficients are estimated iteratively. Parameters ---------- maxiter : integer, optional the number of iterations rtol : float, optional Relative tolerance between estimated coefficients to stop the estimation. Stops if max(abs(last - current) / abs(last)) < rtol iRZRiR%iRtdfthistoryN(RRRRGR%R6RYtappendRZRtmaxRRt yule_walkerRRRtiterRt converged( R-RtrtoltkwdsRRRRtlasttdifft_((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt iterative_fits<     ,       cCswtj|tj}|j}xHt|jD]7}||d|j||d|d !||d)q1W||jS(s Whiten a series of columns according to an AR(p) covariance structure. This drops initial p observations. Parameters ---------- X : array-like The data to be whitened, Returns ------- whitened array ii(RRRtcopyRRR(R-R=t_XR((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR1s  5N( RcRdRRRRReRR*RR1(((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR\s 2=tunbiasedc st|j}|d kr-tdntj|dtj}|r^||j8}n|pn|jd|dkrfd}nfd}|jdkr|jddkrtd ntj |dtj}|d j |d|d>> import statsmodels.api as sm >>> from statsmodels.datasets.sunspots import load >>> data = load(as_pandas=False) >>> rho, sigma = sm.regression.yule_walker(data.endog, ... order=4, method="mle") >>> rho array([ 1.28310031, -0.45240924, -0.20770299, 0.04794365]) >>> sigma 16.808022730464351 Rtmles1ACF estimation method must be 'unbiased' or 'MLE'tdtypeics|S(N((tk(R(sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt>tcsS(N((R(R(sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR@Ris,expecting a vector to estimate AR parametersiiN(RR(tstrtlowerRRRRtmeanRRRRnRRRRNRR( R=RRSRRtdemeantdenomRRRWRtsigmasq((RsBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRs,8  "."c Csddlm}m}tjtj|}|jdkrLtdnt|}|dkrstdn|r||j }n|||d|\}}||\}}||dfS(s Burg's AP(p) parameter estimator Parameters ---------- endog : array-like The endogenous variable order : int, optional Order of the AR. Default is 1. demean : bool, optional Flag indicating to subtract the mean from endog before estimation Returns ------- rho : ndarray AR(p) coefficients computed using Burg's algorithm sigma2 : float Estimate of the residual variance Notes ----- AR model estimated includes a constant estimated using the sample mean. This value is not reported. References ---------- .. [1] Brockwell, P.J. and Davis, R.A., 2016. Introduction to time series and forecasting. Springer. i(tlevinson_durbin_pacft pacf_burgis'endog must be 1-d or squeezable to 1-d.s&order must be an integer larger than 1R( tstatsmodels.tsa.stattoolsRRRRRRRRR( R.RRRRtpacfR tarR((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytburgQs  RQcBseZdZiZeddeedZdZdedZedZ edZ ed Z ed Z e d Zed Zed ZedZedZedZedZedZedZedZedZedZedZedZedZedZedZdZedZ edZ!edZ"ed Z#ed!Z$ed"Z%ed#Z&ed$Z'ed%Z(d&Z)e*e+d'Z,d(Z-e+d)Z.d*ed+Z/ee*eed,Z0e1j0je0_eeedd-Z2eeedd.d/Z3RS(0sK This class summarizes the fit of a linear regression model. It handles the output of contrasts, estimates of covariance, etc. Attributes ---------- pinv_wexog See specific model class docstring cov_HC0 Heteroscedasticity robust covariance matrix. See HC0_se below. cov_HC1 Heteroscedasticity robust covariance matrix. See HC1_se below. cov_HC2 Heteroscedasticity robust covariance matrix. See HC2_se below. cov_HC3 Heteroscedasticity robust covariance matrix. See HC3_se below. cov_type Parameter covariance estimator used for standard errors and t-stats df_model Model degrees of freedom. The number of regressors `p`. Does not include the constant if one is present df_resid Residual degrees of freedom. `n - p - 1`, if a constant is present. `n - p` if a constant is not included. het_scale adjusted squared residuals for heteroscedasticity robust standard errors. Is only available after `HC#_se` or `cov_HC#` is called. See HC#_se for more information. history Estimation history for iterative estimators HC0_se White's (1980) heteroskedasticity robust standard errors. Defined as sqrt(diag(X.T X)^(-1)X.T diag(e_i^(2)) X(X.T X)^(-1) where e_i = resid[i] HC0_se is a cached property. When HC0_se or cov_HC0 is called the RegressionResults instance will then have another attribute `het_scale`, which is in this case is just resid**2. HC1_se MacKinnon and White's (1985) alternative heteroskedasticity robust standard errors. Defined as sqrt(diag(n/(n-p)*HC_0) HC1_see is a cached property. When HC1_se or cov_HC1 is called the RegressionResults instance will then have another attribute `het_scale`, which is in this case is n/(n-p)*resid**2. HC2_se MacKinnon and White's (1985) alternative heteroskedasticity robust standard errors. Defined as (X.T X)^(-1)X.T diag(e_i^(2)/(1-h_ii)) X(X.T X)^(-1) where h_ii = x_i(X.T X)^(-1)x_i.T HC2_see is a cached property. When HC2_se or cov_HC2 is called the RegressionResults instance will then have another attribute `het_scale`, which is in this case is resid^(2)/(1-h_ii). HC3_se MacKinnon and White's (1985) alternative heteroskedasticity robust standard errors. Defined as (X.T X)^(-1)X.T diag(e_i^(2)/(1-h_ii)^(2)) X(X.T X)^(-1) where h_ii = x_i(X.T X)^(-1)x_i.T HC3_see is a cached property. When HC3_se or cov_HC3 is called the RegressionResults instance will then have another attribute `het_scale`, which is in this case is resid^(2)/(1-h_ii)^(2). model A pointer to the model instance that called fit() or results. params The linear coefficients that minimize the least squares criterion. This is usually called Beta for the classical linear model. resid_pearson `wresid` normalized to have unit variance. g?R?c KsEtt|j||||i|_t|drF|j|_n d|_|j|_|j |_ |dkrd|_ idddd6|_ |dkrt }n||_ ng|dkri}nd|kr|jd} |dkr| }qn|jd|d t d||x"|D]} t|| || q#WdS( NRJR?s Standard Errors assume that the s-covariance matrix of the errors is correctly s specified.t descriptionRFRDtuse_self(R)RQR*t_cacheRGRJt_wexog_singular_valuesRR9R;RDRERRFtpoptget_robustcov_resultstsetattr( R-tmodelRZR@R^RDRERFR0tuse_t_2tkey((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR*s4                cCs|jdS(N(tsummary(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt__str__sg?cCs%tt|jd|d|}|S(s Returns the confidence interval of the fitted parameters. Parameters ---------- alpha : float, optional The `alpha` level for the confidence interval. ie., The default `alpha` = .05 returns a 95% confidence interval. cols : array-like, optional `cols` specifies which confidence intervals to return Notes ----- The confidence interval is based on Student's t-distribution. R|tcols(R)RQtconf_int(R-R|R tci((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRs!cCst|jjjdS(sNumber of observations n.i(R2RR'R(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR! scCs|jj|j|jjS(s:The predicted values for the original (unwhitened) design.(RR[RZR/(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt fittedvaluesscCs&|jj|jj|j|jjS(sW The residuals of the transformed/whitened regressand and regressor(s) (RR&R[RZR'(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytwresidscCs&|jj|jj|j|jjS(sThe residuals of the model.(RR.R[RZR/(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRscCs |j}tj|||jS(s A scale factor for the covariance matrix. Default value is ssr/(n-p). Note that the square root of `scale` is often called the standard error of the regression. (RRRHR;(R-R((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR^"s cCs|j}tj||S(s$Sum of squared (whitened) residuals.(RRRH(R-R((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR,s cCs|j}t|dd}t|dd}|dk rmtj|jd|}tj||j|dS|dk rtj|j}|j|}|j j ||j |}|j|}|j|}tj|dS|j |j j }tj ||SdS(s<The total (weighted) sum of squares centered about the mean.R(R iN( RtgetattrRRtaverageR.Rnt ones_likeR1R&RHR(R-RR(R Rtiotaterrtcentered_endog((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt centered_tss2s    cCs|jj}tj||S(s Uncentered sum of squares. Sum of the squared values of the (whitened) endogenous response variable. (RR&RRH(R-R&((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytuncentered_tssGs cCs)|jr|j|jS|j|jSdS(sExplained sum of squares. If a constant is present, the centered total sum of squares minus the sum of squared residuals. If there is no constant, the uncentered total sum of squares is used.N(R8RRR(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytessPs cCs1|jrd|j|jSd|j|jSdS(s R-squared of a model with an intercept. This is defined here as 1 - `ssr`/`centered_tss` if the constant is included in the model and 1 - `ssr`/`uncentered_tss` if the constant is omitted. iN(R8RRR(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytrsquared[s cCs,dtj|j|j|jd|jS(s Adjusted R-squared. This is defined here as 1 - (`nobs`-1)/`df_resid` * (1-`rsquared`) if a constant is included and 1 - `nobs`/`df_resid` * (1-`rsquared`) if no constant is included. i(RtdivideR!R8R;R(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt rsquared_adjhscCs|j|jS(s Mean squared error the model. This is the explained sum of squares divided by the model degrees of freedom. (RR9(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt mse_modelsscCs|j|jS(s Mean squared error of the residuals. The sum of squared residuals divided by the residual degrees of freedom. (RR;(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt mse_resid{scCs7|jr|j|j|jS|j|j|jSdS(s Total mean squared error. Defined as the uncentered total sum of squares divided by n the number of observations. N(R8RR;R9R(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt mse_totals cCst|dr|jdkr|jjd}tj|}|jjj}|jjj dkr|dkrttj St |}|j |||}|jdkrtj Sn|j|}|j|jd<|jS|j|jSdS(sF-statistic of the fully specified model. Calculated as the mean squared error of the model divided by the mean squared error of the residuals.RDR?iitf_pvalueN(RGRDR@RRteyeRtdatat const_idxR8RtnanRRRtf_testtpvalueRtfvalueRR(R-tk_paramstmatR$tidxtft((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR(s      cCstjj|j|j|jS(sp-value of the F-statistic(RtftsfR(R9R;(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR!scCstjtj|jS(s/The standard errors of the parameter estimates.(RRRKt cov_params(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytbsescCsd|jd|j|jS(sAkaike's information criteria. For a model with a constant :math:`-2llf + 2(df\_model + 1)`. For a model without a constant :math:`-2llf + 2(df\_model)`.ii(RuR9R8(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytaicscCs)d|jtj|j|j|jS(sBayes' information criteria. For a model with a constant :math:`-2llf + \log(n)(df\_model+1)`. For a model without a constant :math:`-2llf + \log(n)(df\_model)`i(RuRRoR!R9R8(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytbicscCsi|jdk r|jd}n0tjjjtj|jjj|jj}tj |dddS(s@ Return eigenvalues sorted in decreasing order. iNi( RRRRteigvalshRHRR'Rtsort(R-teigvals((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt eigenvalss cCs"|j}tj|d|dS(s~ Return condition number of exogenous matrix. Calculated as ratio of largest to smallest eigenvalue. ii(R6RR(R-R5((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytcondition_numbers cCs9tj|jj|dddf|jjj}|S(N(RRHRR%RR(R-R^tH((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt_HCCMs&cCs&|jd|_|j|j}|S(s3 See statsmodels.RegressionResults i(Rt het_scaleR9(R-tcov_HC0((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR;scCs4|j|j|jd|_|j|j}|S(s3 See statsmodels.RegressionResults i(R!R;RR:R9(R-tcov_HC1((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR<scCs[tjt|jj|j|jjj}|jdd||_|j |j}|S(s3 See statsmodels.RegressionResults ii( RRKRRR'R@RRR:R9(R-thtcov_HC2((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR>s cCs[tjt|jj|j|jjj}|jd|d|_|j |j}|S(s3 See statsmodels.RegressionResults ii( RRKRRR'R@RRR:R9(R-R=tcov_HC3((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR?s  $cCstjtj|jS(s3 See statsmodels.RegressionResults (RRRKR;(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytHC0_sescCstjtj|jS(s3 See statsmodels.RegressionResults (RRRKR<(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytHC1_sescCstjtj|jS(s3 See statsmodels.RegressionResults (RRRKR>(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytHC2_sescCstjtj|jS(s3 See statsmodels.RegressionResults (RRRKR?(R-((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytHC3_se%scCst|dstdntj|jjj}tj|jd||j j j krddl m }|dt|jS|jtj|jSdS(s Residuals, normalized to have unit variance. Returns ------- An array wresid standardized by the sqrt if scale RsMethod requires residuals.i i(twarns5All residuals are 0, cannot compute normed residuals.N(RGRRtfinfoRRtepsRR^RR.RtwarningsRDtRuntimeWarning(R-RFRD((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt resid_pearson,s ,cCs|jj|jjkrtS|jj}|jj}||krDtS|jj}|j}||dddf}tjtj |j dd}tj |dS(s Parameters ---------- restricted : Result instance The restricted model is assumed to be nested in the current model. The result instance of the restricted model is required to have two attributes, residual sum of squares, `ssr`, residual degrees of freedom, `df_resid`. Returns ------- nested : bool True if nested, otherwise false Notes ----- A most nests another model if the regressors in the smaller model are spanned by the regressors in the larger model and the regressand is identical. Nii( RR!RR5R'RRRRRtallclose(R-t restrictedt full_ranktrestricted_ranktrestricted_exogt full_wresidtscorestscore_l2((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt _is_nestedBs     %cCs@ddljj}ddlm}|j|s@tdn|j}|jj }||dddf}|j } |j } |j } | | } |j dd} |r||jdddf}t}n|r||j ddddf}nt|dd}|dkr[tj |d }tj|j|| }|||}n|dkr|tj|j|| }n||dkr|jd}||j||| }nD|dkr|jd}||j||}ntdd| t| || j}tjj|| }||| fS(s'Use Lagrange Multiplier test to test whether restricted model is correct Parameters ---------- restricted : Result instance The restricted model is assumed to be nested in the current model. The result instance of the restricted model is required to have two attributes, residual sum of squares, `ssr`, residual degrees of freedom, `df_resid`. demean : bool Flag indicating whether the demean the scores based on the residuals from the restricted model. If True, the covariance of the scores are used and the LM test is identical to the large sample version of the LR test. Returns ------- lm_value : float test statistic, chi2 distributed p_value : float p-value of the test statistic df_diff : int degrees of freedom of the restriction, i.e. difference in df between models Notes ----- TODO: explain LM text iN(Rs-Restricted model is not nested by full model.RmiRDR?itHC0tHC1tHC2tHC3tHACtmaxlagstclustertgroupss(Only nonrobust, HC, HAC and cluster are scurrently connected(RSRTRURV(t%statsmodels.stats.sandwich_covarianceRtsandwich_covariancet numpy.linalgRRRRRRR'RR!R;RRRRRHRREt S_hac_simpletS_crosssectionRtchi2R.(R-RKRtuse_lrtswRRR'RPRtdf_fulltdf_restrtdf_diffRRDtsigma2tXpXtSinvRXRZtlm_valuetp_value((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytcompare_lm_testisF       &  "     c Cst|dddk}t|dddk}|s<|rStjddtn|j}|j}|j}|j}||}|||||} tjj| ||} | | |fS(sluse F test to test whether restricted model is correct Parameters ---------- restricted : Result instance The restricted model is assumed to be nested in the current model. The result instance of the restricted model is required to have two attributes, residual sum of squares, `ssr`, residual degrees of freedom, `df_resid`. Returns ------- f_value : float test statistic, F distributed p_value : float p-value of the test statistic df_diff : int degrees of freedom of the restriction, i.e. difference in df between models Notes ----- See mailing list discussion October 17, This test compares the residual sum of squares of the two models. This is not a valid test, if there is unspecified heteroscedasticity or correlation. This method will issue a warning if this is detected but still return the results under the assumption of homoscedasticity and no autocorrelation (sphericity). RDR?s-F test for comparison is likely invalid with s$robust covariance, proceeding anyway( RRGRDR RR;RR-R.( R-RKt has_robust1t has_robust2tssr_fullt ssr_restrRcRdRetf_valueRj((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytcompare_f_tests!         c Cs|r|j|dtSt|dddk}t|dddk}|sU|rltjddtn|j}|j}|j}|j}||} d||} tj j | | } | | | fS(s Likelihood ratio test to test whether restricted model is correct Parameters ---------- restricted : Result instance The restricted model is assumed to be nested in the current model. The result instance of the restricted model is required to have two attributes, residual sum of squares, `ssr`, residual degrees of freedom, `df_resid`. large_sample : bool Flag indicating whether to use a heteroskedasticity robust version of the LR test, which is a modified LM test. Returns ------- lr_stat : float likelihood ratio, chisquare distributed with df_diff degrees of freedom p_value : float p-value of the test statistic df_diff : int degrees of freedom of the restriction, i.e. difference in df between models Notes ----- The exact likelihood ratio is valid for homoskedastic data, and is defined as .. math:: D=-2\log\left(\frac{\mathcal{L}_{null}} {\mathcal{L}_{alternative}}\right) where :math:`\mathcal{L}` is the likelihood of the model. With :math:`D` distributed as chisquare with df equal to difference in number of parameters or equivalently difference in residual degrees of freedom. The large sample version of the likelihood ratio is defined as .. math:: D=n s^{\prime}S^{-1}s where :math:`s=n^{-1}\sum_{i=1}^{n} s_{i}` .. math:: s_{i} = x_{i,alternative} \epsilon_{i,null} is the average score of the model evaluated using the residuals from null model and the regressors from the alternative model and :math:`S` is the covariance of the scores, :math:`s_{i}`. The covariance of the scores is estimated using the same estimator as in the alternative model. This test compares the loglikelihood of the two models. This may not be a valid test, if there is unspecified heteroscedasticity or correlation. This method will issue a warning if this is detected but still return the results without taking unspecified heteroscedasticity or correlation into account. This test compares the loglikelihood of the two models. This may not be a valid test, if there is unspecified heteroscedasticity or correlation. This method will issue a warning if this is detected but still return the results without taking unspecified heteroscedasticity or correlation into account. is the average score of the model evaluated using the residuals from null model and the regressors from the alternative model and :math:`S` is the covariance of the scores, :math:`s_{i}`. The covariance of the scores is estimated using the same estimator as in the alternative model. TODO: put into separate function, needs tests RaRDR?s-Likelihood Ratio test is likely invalid with s$robust covariance, proceeding anywayi( RkRRRGRDR RuR;RR`R.( R-RKt large_sampleRlRmtllf_fullt llf_restrRcRdtlrdftlrstatt lr_pvalue((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytcompare_lr_tests R        RTcKsuddljj}ddlm}m}||}d|krV|jd|d7}|d?7}||d}|j |nJ|d@krdA}|dB7}|dC7}|dD7}||}|j |n|rAgt|D]"\}}dEj |d9|^q}|j!d8dF|j"|n|S(PsSummarize the Regression Results Parameters ---------- yname : string, optional Default is `y` xname : list of strings, optional Default is `var_##` for ## in p the number of regressors title : string, optional Title for the top table. If not None, then this replaces the default title alpha : float significance level for the confidence intervals Returns ------- smry : Summary instance this holds the summary tables and text, which can be printed or converted to various output formats. See Also -------- statsmodels.iolib.summary.Summary : class to hold summary results i(t jarque_berat omni_normtestt durbin_watsontjbtjbpvtskewtkurtosistomnitomnipvtcondnot mineigvalsDep. Variable:sModel:sMethod:s Least SquaressDate:sTime:sNo. Observations:s Df Residuals:s Df Model:RDsCovariance Type:Rs (uncentered)s R-squaredt:s%#8.3fsAdj. R-squareds F-statistic:s%#8.4gsProb (F-statistic):s%#6.3gsLog-Likelihood:sAIC:sBIC:sOmnibus:s%#6.3fsProb(Omnibus):sSkew:s Kurtosis:sDurbin-Watson:sJarque-Bera (JB):s Prob(JB):s%#8.3gs Cond. No.t sRegression Results(tSummarytglefttgrighttynametxnamettitleR|RFRiis9The input rank is higher than the number of observations.g|=s6The smallest eigenvalue is %6.3g. This might indicate sthat there are s5strong multicollinearity problems or that the design smatrix is singular.is1The condition number is large, %6.3g. This might sindicate that there are s,strong multicollinearity or other numerical s problems.s [{0}] {1}s Warnings:N(sDep. Variable:N(sModel:N(sDate:N(sTime:N(sNo. Observations:N(s Df Residuals:N(s Df Model:N(sLog-Likelihood:N(#tstatsmodels.stats.stattoolsRRRRR6R7tdicttdiagnRRGRRDR8RRR(R!R1R2RRRctstatsmodels.iolib.summaryRtadd_table_2colstadd_table_paramsRFRER/Rt enumerateRtinsertt add_extra_txt(R-RRRR|RRRRRRRRRR5Rttop_leftt rsquared_typet top_rightt diagn_leftt diagn_rightRtsmrytetexttwstrRttext((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR x s         &        2s%.4fc Csddlm}m}m}ddlm} ||j\} } } } ||j\}}||j}|j}|j}t j |}| dd|fdd|fdd| fdd| fd d|fd d| fd d| fd d |fg}ddl m }|j }|jd|d|d|d|d|d||j||ddkrd|d}|j|n|dkrd|}|j|n|S(sExperimental summary function to summarize the regression results Parameters ---------- xname : List of strings of length equal to the number of parameters Names of the independent variables (optional) yname : string Name of the dependent variable (optional) title : string, optional Title for the top table. If not None, then this replaces the default title alpha : float significance level for the confidence intervals float_format: string print format for floats in parameters summary Returns ------- smry : Summary instance this holds the summary tables and text, which can be printed or converted to various output formats. See Also -------- statsmodels.iolib.summary2.Summary : class to hold summary results i(RRR(t OrderedDictsOmnibus:s%.3fsProb(Omnibus):sSkew:s Kurtosis:sDurbin-Watson:sJarque-Bera (JB):s Prob(JB):sCondition No.:s%.0f(tsummary2RR|t float_formatRRRg|=sThe smallest eigenvalue is %6.3g. This might indicate that there are strong multicollinearity problems or that the design matrix is singular.is|* The condition number is large (%.g). This might indicate strong multicollinearity or other numerical problems.(RRRRt collectionsRRR6R7RR4tstatsmodels.iolibRRtadd_basetadd_dicttadd_text(R-RRRR|RRRRRRRRRRRtdwR5Rt diagnosticRRRD((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR s:             (4RcRdReRRR*R RR R!RRRR R^RRRRRRRRR R(R!R0R1R2R6R7R9R;R<R>R?R@RARBRCRIRRRRRkRqRxRRRR R(((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRQsbJ$              'Q 4 i   RPcBs_eZdZdZddd ed dZddddddZdd d ddd ZRS( s2 Results class for for an OLS model. Most of the methods and attributes are inherited from RegressionResults. The special methods that are only available for OLS are: - get_influence - outlier_test - el_test - conf_int_el See Also -------- RegressionResults cCsddlm}||S(so get an instance of Influence with influence and outlier measures Returns ------- infl : Influence instance the instance has methods to calculate the main influence and outlier measures for the OLS regression See Also -------- statsmodels.stats.outliers_influence.OLSInfluence i(t OLSInfluence(t$statsmodels.stats.outliers_influenceR(R-R((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt get_influenceT stbonfg?c Cs2ddlm}||||d|d|d|S(sm Test observations for outliers according to method Parameters ---------- method : str - `bonferroni` : one-step correction - `sidak` : one-step correction - `holm-sidak` : - `holm` : - `simes-hochberg` : - `hommel` : - `fdr_bh` : Benjamini/Hochberg - `fdr_by` : Benjamini/Yekutieli See `statsmodels.stats.multitest.multipletests` for details. alpha : float familywise error rate labels : None or array_like If `labels` is not None, then it will be used as index to the returned pandas DataFrame. See also Returns below order : bool Whether or not to order the results by the absolute value of the studentized residuals. If labels are provided they will also be sorted. cutoff : None or float in [0, 1] If cutoff is not None, then the return only includes observations with multiple testing corrected p-values strictly below the cutoff. The returned array or dataframe can be empty if t Returns ------- table : ndarray or DataFrame Returns either an ndarray or a DataFrame if labels is not None. Will attempt to get labels from model_results if available. The columns are the Studentized residuals, the unadjusted p-value, and the corrected p-value according to method. Notes ----- The unadjusted p-value is stats.t.sf(abs(resid), df) where df = df_resid - 1. i(t outlier_testtlabelsRtcutoff(RR(R-RSR|RRRR((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRe s-itnmicCstj|j}t} t|t|kr| jgd|d|jjd|jjd|jj d|jjj dd|d|d |} dt j j | t|} |r| | | jfS| | fSntj||} ||jj|jj|jj |jjj d|||f} |d krhtj| j| d d d d ddddd| d} n|dkrtj| j| ddddd| d} ndt j j | t|} |r| | | j| jfS|r| | | jfS| | fSdS(s Tests single or joint hypotheses of the regression parameters using Empirical Likelihood. Parameters ---------- b0_vals : 1darray The hypothesized value of the parameter to be tested param_nums : 1darray The parameter number to be tested print_weights : bool If true, returns the weights that optimize the likelihood ratio at b0_vals. Default is False ret_params : bool If true, returns the parameter vector that maximizes the likelihood ratio at b0_vals. Also returns the weights. Default is False method : string Can either be 'nm' for Nelder-Mead or 'powell' for Powell. The optimization method that optimizes over nuisance parameters. Default is 'nm' stochastic_exog : bool When TRUE, the exogenous variables are assumed to be stochastic. When the regressors are nonstochastic, moment conditions are placed on the exogenous variables. Confidence intervals for stochastic regressors are at least as large as non-stochastic regressors. Default = TRUE Returns ------- res : tuple The p-value and -2 times the log-likelihood ratio for the hypothesized values. Examples -------- >>> import statsmodels.api as sm >>> data = sm.datasets.stackloss.load(as_pandas=False) >>> endog = data.endog >>> exog = sm.add_constant(data.exog) >>> model = sm.OLS(endog, exog) >>> fitted = model.fit() >>> fitted.params >>> array([-39.91967442, 0.7156402 , 1.29528612, -0.15212252]) >>> fitted.rsquared >>> 0.91357690446068196 >>> # Test that the slope on the first variable is 0 >>> fitted.el_test([0], [1]) >>> (27.248146353888796, 1.7894660442330235e-07) t param_numsR.R/R!tnvariRZtb0_valststochastic_exogRtmaxfuni'Rt full_outputtdispitargstpowellN(RRRZR Rlt_opt_nuis_regressRR.R/R!RRR`tcdft new_weightstdeleteRtfmint fmin_powellt new_params(R-RRtreturn_weightst ret_paramsRSRt return_paramsRZt opt_fun_insttllrtpvaltx0R((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pytel_test sF;          c stjjd|d|dkr?jdd}n|dkrejdd}nfd}tj||j}tj|j|} || fS(s Computes the confidence interval for the parameter given by param_num using Empirical Likelihood Parameters ---------- param_num : float The parameter for which the confidence interval is desired sig : float The significance level. Default is .05 upper_bound : float The maximum value the upper limit can be. Default is the 99.9% confidence value under OLS assumptions. lower_bound : float The minimum value the lower limit can be. Default is the 99.9% confidence value under OLS assumptions. method : string Can either be 'nm' for Nelder-Mead or 'powell' for Powell. The optimization method that optimizes over nuisance parameters. Default is 'nm' Returns ------- ci : tuple The confidence interval See Also -------- el_test Notes ----- This function uses brentq to find the value of beta where test_beta([beta], param_num)[1] is equal to the critical value. The function returns the results of each iteration of brentq at each value of beta. The current function value of the last printed optimization should be the critical value at the desired significance level. For alpha=.05, the value is 3.841459. To ensure optimization terminated successfully, it is suggested to do el_test([lower_limit], [param_num]) If the optimization does not terminate successfully, consider switching optimization algorithms. If optimization is still not successful, try changing the values of start_int_params. If the current function value repeatedly jumps from a number between 0 and the critical value and a very large number (>50), the starting parameters of the interior minimization need to be changed. ig{Gz?ics<jtj|gtjgdddS(NRSRi(RRR(tb0(RSt param_numtr0R-R(sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyR> s'N(RR`tppfRRRtbrenthRZ( R-Rtsigt upper_boundt lower_boundRSRR-tlowerltupperl((RSRRR-RsBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt conf_int_el sA    N( RcRdReRRRRRR(((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRPB s  0`RRcBseZi dd6dd6dd6dd6dd6dd6dd 6dd 6dd 6dd 6dd 6ZejejjeZiZejejj eZ RS(tcolumnstchisqtrowstsresidR(Rtcovt bcov_unscaledt bcov_scaledR@RARBRCt norm_resid( RcRdt_attrstwrapt union_dictsRtLikelihoodResultsWrappert _wrap_attrst_methodst _wrap_methods(((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyRRI s$   (9Ret __future__Rtstatsmodels.compat.pythonRRRtnumpyRt scipy.linalgRtscipyRRtstatsmodels.tools.toolsRRtstatsmodels.tools.decoratorsR R tstatsmodels.base.modelRRtstatsmodels.base.wrappertwrapperR tstatsmodels.emplike.elregressR RGtstatsmodels.tools.sm_exceptionsR t"statsmodels.regression._predictionR RRRt __docformat__t__all__RR#tLikelihoodModelR$RRRRRRRRRtLikelihoodModelResultsRQRPtResultsWrapperRRtpopulate_wrapper(((sBlib/python2.7/site-packages/statsmodels/regression/linear_model.pyt sL  g F  S.