p7]c@sdZddlZddlmZddlmZddljj Zddl j j Z ddljjZddlj jZddlmZmZmZddlmZddlmZdd lmZmZmZdd l m!Z!d d gZ"d Z#d ej$fdYZ%dej&fdYZ'de j(fdYZ)ej*e)e'e+dkrddl,j-Z.e.j/j0j1de2Z3e%e3j4e3j5j6Z7e7j8ddZ9e7j8ddZ:ndS(s Generalized linear models currently supports estimation using the one-parameter exponential families References ---------- Gill, Jeff. 2000. Generalized Linear Models: A Unified Approach. SAGE QASS Series. Green, PJ. 1984. "Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives." Journal of the Royal Statistical Society, Series B, 46, 149-192. Hardin, J.W. and Hilbe, J.M. 2007. "Generalized Linear Models and Extensions." 2nd ed. Stata Press, College Station, TX. McCullagh, P. and Nelder, J.A. 1989. "Generalized Linear Models." 2nd ed. Chapman & Hall, Boca Rotan. iNi(tfamilies(tcache_readonly(t_plot_added_variable_doct_plot_partial_residuals_doct_plot_ceres_residuals_doc(t _prediction(tPredictionResults(tPerfectSeparationErrort DomainWarningtHessianInversionWarning(t LinAlgErrortGLMRcCs(tj||||dd|d|S(Nitatoltrtol(tnptallclose(t criteriont iterationR R ((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyt_check_convergence/sc BseZdiejd6Zd-d-d-d-d-ddZdZdZdZ ddZ d-d Z d-d Z d-d Z d-d Zd-ed Zd-d-dZd-dZdZd-dZd-d-edZdZdZdZddddZd-d-d-edZdd-d-d-dZdZd-ddd d-d!d-d-eed"d# Zd-d$dd eed-d!d-d-d"d% Zd-dd d-d!d-d-d&Z d'd(d-ed)Z!d*d+Z"d-d,Z#RS(.s& Generalized Linear Models class GLM inherits from statsmodels.base.model.LikelihoodModel Parameters ---------- endog : array-like 1d array of endogenous response variable. This array can be 1d or 2d. Binomial family models accept a 2d array with two columns. If supplied, each observation is expected to be [success, failure]. exog : array-like A nobs x k array where `nobs` is the number of observations and `k` is the number of regressors. An intercept is not included by default and should be added by the user (models specified using a formula include an intercept by default). See `statsmodels.tools.add_constant`. family : family class instance The default is Gaussian. To specify the binomial distribution family = sm.family.Binomial() Each family can take a link instance as an argument. See statsmodels.family.family for more information. offset : array-like or None An offset to be included in the model. If provided, must be an array whose length is the number of rows in exog. exposure : array-like or None Log(exposure) will be added to the linear prediction in the model. Exposure is only valid if the log link is used. If provided, it must be an array with the same length as endog. freq_weights : array-like 1d array of frequency weights. The default is None. If None is selected or a blank value, then the algorithm will replace with an array of 1's with length equal to the endog. WARNING: Using weights is not verified yet for all possible options and results, see Notes. var_weights : array-like 1d array of variance (analytic) weights. The default is None. If None is selected or a blank value, then the algorithm will replace with an array of 1's with length equal to the endog. WARNING: Using weights is not verified yet for all possible options and results, see Notes. %(extra_params)s Attributes ---------- df_model : float Model degrees of freedom is equal to p - 1, where p is the number of regressors. Note that the intercept is not reported as a degree of freedom. df_resid : float Residual degrees of freedom is equal to the number of observation n minus the number of regressors p. endog : array See Notes. Note that `endog` is a reference to the data so that if data is already an array and it is changed, then `endog` changes as well. exposure : array-like Include ln(exposure) in model with coefficient constrained to 1. Can only be used if the link is the logarithm function. exog : array See Notes. Note that `exog` is a reference to the data so that if data is already an array and it is changed, then `exog` changes as well. freq_weights : array See Notes. Note that `freq_weights` is a reference to the data so that if data is already an array and it is changed, then `freq_weights` changes as well. var_weights : array See Notes. Note that `var_weights` is a reference to the data so that if data is already an array and it is changed, then `var_weights` changes as well. iteration : int The number of iterations that fit has run. Initialized at 0. family : family class instance The distribution family of the model. Can be any family in statsmodels.families. Default is Gaussian. mu : array The mean response of the transformed variable. `mu` is the value of the inverse of the link function at lin_pred, where lin_pred is the linear predicted value of the WLS fit of the transformed variable. `mu` is only available after fit is called. See statsmodels.families.family.fitted of the distribution family for more information. n_trials : array See Notes. Note that `n_trials` is a reference to the data so that if data is already an array and it is changed, then `n_trials` changes as well. `n_trials` is the number of binomial trials and only available with that distribution. See statsmodels.families.Binomial for more information. normalized_cov_params : array The p x p normalized covariance of the design / exogenous data. This is approximately equal to (X.T X)^(-1) offset : array-like Include offset in model with coefficient constrained to 1. scale : float The estimate of the scale / dispersion of the model fit. Only available after fit is called. See GLM.fit and GLM.estimate_scale for more information. scaletype : str The scaling used for fitting the model. This is only available after fit is called. The default is None. See GLM.fit for more information. weights : array The value of the weights after the last iteration of fit. Only available after fit is called. See statsmodels.families.family for the specific distribution weighting functions. Examples -------- >>> import statsmodels.api as sm >>> data = sm.datasets.scotland.load(as_pandas=False) >>> data.exog = sm.add_constant(data.exog) Instantiate a gamma family model with the default link function. >>> gamma_model = sm.GLM(data.endog, data.exog, ... family=sm.families.Gamma()) >>> gamma_results = gamma_model.fit() >>> gamma_results.params array([-0.01776527, 0.00004962, 0.00203442, -0.00007181, 0.00011185, -0.00000015, -0.00051868, -0.00000243]) >>> gamma_results.scale 0.0035842831734919055 >>> gamma_results.deviance 0.087388516416999198 >>> gamma_results.pearson_chi2 0.086022796163805704 >>> gamma_results.llf -83.017202161073527 See Also -------- statsmodels.genmod.families.family.Family :ref:`families` :ref:`links` Notes ----- Only the following combinations make sense for family and link: ============= ===== === ===== ====== ======= === ==== ====== ====== ==== Family ident log logit probit cloglog pow opow nbinom loglog logc ============= ===== === ===== ====== ======= === ==== ====== ====== ==== Gaussian x x x x x x x x x inv Gaussian x x x binomial x x x x x x x x x Poission x x x neg binomial x x x x gamma x x x Tweedie x x x ============= ===== === ===== ====== ======= === ==== ====== ====== ==== Not all of these link functions are currently available. Endog and exog are references so that if the data they refer to are already arrays and these arrays are changed, endog and exog will change. Statsmodels supports two separte definitions of weights: frequency weights and variance weights. Frequency weights produce the same results as repeating observations by the frequencies (if those are integers). Frequency weights will keep the number of observations consistent, but the degrees of freedom will change to reflect the new weights. Variance weights (referred to in other packages as analytic weights) are used when ``endog`` represents an an average or mean. This relies on the assumption that that the inverse variance scales proportionally to the weight--an observation that is deemed more credible should have less variance and therefore have more weight. For the ``Poisson`` family--which assumes that occurences scale proportionally with time--a natural practice would be to use the amount of time as the variance weight and set ``endog`` to be a rate (occurrances per period of time). Similarly, using a compound Poisson family, namely ``Tweedie``, makes a similar assumption about the rate (or frequency) of occurences having variance proportional to time. Both frequency and variance weights are verified for all basic results with nonrobust or heteroscedasticity robust ``cov_type``. Other robust covariance types have not yet been verified, and at least the small sample correction is currently not based on the correct total frequency count. Currently, all residuals are not weighted by frequency, although they may incorporate ``n_trials`` for ``Binomial`` and ``var_weights`` +---------------+----------------------------------+ | Residual Type | Applicable weights | +===============+==================================+ | Anscombe | ``var_weights`` | +---------------+----------------------------------+ | Deviance | ``var_weights`` | +---------------+----------------------------------+ | Pearson | ``var_weights`` and ``n_trials`` | +---------------+----------------------------------+ | Reponse | ``n_trials`` | +---------------+----------------------------------+ | Working | ``n_trials`` | +---------------+----------------------------------+ WARNING: Loglikelihood and deviance are not valid in models where scale is equal to 1 (i.e., ``Binomial``, ``NegativeBinomial``, and ``Poisson``). If variance weights are specified, then results such as ``loglike`` and ``deviance`` are based on a quasi-likelihood interpretation. The loglikelihood is not correctly specified in this case, and statistics based on it, such AIC or likelihood ratio tests, are not appropriate. t extra_paramstnonec KsK|dk r`t|jt|j r`ddl} | jd|jjj|jjft n|dk r~t j |}n|dk rt j |}n|dk rt j |}n|dk rt j |}n||_ ||_tt|j||d|d|d|d|d|| |j||j|j|j|j |j|dkrkt|dn|dkrt|dn|jjd|_|jjd d ddd d d g|jjd|jd | kr| d |_nd} t|dr|j} nt|dr5| |j} n| |_ d|_!dS(NisBThe %s link function does not respect the domain of the %s family.tmissingtoffsettexposuret freq_weightst var_weightsitweightstmutiweightst_offset_exposuretn_trialstfamilyg("tNonet isinstancetlinkttuplet safe_linkstwarningstwarnt __class__t__name__RRtlogtasarrayRRtsuperR t__init__t _check_inputsRRtendogtdelattrtshapetnobst _data_attrtextendt _init_keystappendt_setup_binomialRthasattrRt scaletype( tselfR.texogRRRRRRtkwargsR%toffset_exposure((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyR,sT                     cCstjj|jd|_|jdk rw|jjd|jjdkrw|jj |_ |j |jd|_ n1|jjd|_ |jjd|jd|_ dS(s8 Initialize a generalized linear model. iiN( Rtlinalgt matrix_rankR:tdf_modelRR R0R.tsumtwnobstdf_resid(R9((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyt initializeDs cCsT|dkrtj}n||_|dk rt|jjtjjsZtdq|j d|j dkrtdqn|dk r|j d|j dkrtdqn|dk r|j d|j dkrtdnt |j dkrtdqn|j dk |_ |j dkrWt j|j d|_ nt j |j d kr|j dkr|j t j|j d|_ n|dk r|j d|j dkrtdnt |j dkrtd qn|dk |_|dkr4t j|j d|_nt j|j |j|_dS( Ns4exposure can only be used with the log link functionis(exposure is not the same length as endogs&offset is not the same length as endogs)freq weights not the same length as endogis$freq weights has too many dimensionss(var weights not the same length as endogs#var weights has too many dimensions((R RtGaussianRR!R"tlinkstLogt ValueErrorR0tlenRt_has_freq_weightsRtonest_has_var_weightsRR*R(R9RRRR.RR((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyR-Rs>     '  cCsOtt|j}d|krK|ddk rKtj|d|dt nonrobustic Ks||_|jdkrm|jdkr6d}n|jd|d|d|d|d|d |d || S| jd |_tj|j|_|j d|d |d|d|d|d | d| d|d |d |d| | } |`|`| SdS(s Fits a generalized linear model for a given family. Parameters ---------- start_params : array-like, optional Initial guess of the solution for the loglikelihood maximization. The default is family-specific and is given by the ``family.starting_mu(endog)``. If start_params is given then the initial mean will be calculated as ``np.dot(exog, start_params)``. maxiter : int, optional Default is 100. method : string Default is 'IRLS' for iteratively reweighted least squares. Otherwise gradient optimization is used. tol : float Convergence tolerance. Default is 1e-8. scale : string or float, optional `scale` can be 'X2', 'dev', or a float The default value is None, which uses `X2` for Gamma, Gaussian, and Inverse Gaussian. `X2` is Pearson's chi-square divided by `df_resid`. The default is 1 for the Binomial and Poisson families. `dev` is the deviance divided by df_resid cov_type : string The type of parameter estimate covariance matrix to compute. cov_kwds : dict-like Extra arguments for calculating the covariance of the parameter estimates. use_t : bool If True, the Student t-distribution is used for inference. full_output : bool, optional Set to True to have all available output in the Results object's mle_retvals attribute. The output is dependent on the solver. See LikelihoodModelResults notes section for more information. Not used if methhod is IRLS. disp : bool, optional Set to True to print convergence messages. Not used if method is IRLS. max_start_irls : int The number of IRLS iterations used to obtain starting values for gradient optimization. Only relevant if `method` is set to something other than 'IRLS'. atol : float, optional (available with IRLS fits) The absolute tolerance criterion that must be satisfied. Defaults to ``tol``. Convergence is attained when: :math:`rtol * prior + atol > abs(current - prior)` rtol : float, optional (available with IRLS fits) The relative tolerance criterion that must be satisfied. Defaults to 0 which means ``rtol`` is not used. Convergence is attained when: :math:`rtol * prior + atol > abs(current - prior)` tol_criterion : str, optional (available with IRLS fits) Defaults to ``'deviance'``. Can optionally be ``'params'``. wls_method : str, optional (available with IRLS fits) options are 'lstsq', 'pinv' and 'qr' specifies which linear algebra function to use for the irls optimization. Default is `lstsq` which uses the same underlying svd based approach as 'pinv', but is faster during iterations. 'lstsq' and 'pinv' regularize the estimate in singular and near-singular cases by truncating small singular values based on `rcond` of the respective numpy.linalg function. 'qr' is only valied for cases that are not singular nor near-singular. optim_hessian : {'eim', 'oim'}, optional (available with scipy optimizer fits) When 'oim'--the default--the observed Hessian is used in fitting. 'eim' is the expected Hessian. This may provide more stable fits, but adds assumption that the Hessian is correctly specified. Notes ----- If method is 'IRLS', then an additional keyword 'attach_wls' is available. This is currently for internal use only and might change in future versions. If attach_wls' is true, then the final WLS instance of the IRLS iteration is attached to the results instance as `results_wls` attribute. tirlsRhRt start_paramstmaxiterttolRPtcov_typetcov_kwdstuse_tt optim_hessianRt full_outputtdisptmax_start_irlsN( R8Rt _fit_irlstgetRgRRnR:Rit _fit_gradient(R9RRRRRPRRRRRRR;tfit_((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyRs*R      tnewtonc Ks7|j} d|_| dkrr|dkrr|jd|d| d|ddddd dd d| }|j}~ntt|jd|d|d|d |d |d || }| |_|j|j}|j|}|j dkrd}n |j |}|j dkrt }d}nt }y-t jj|j|jd| |}Wn4tk rddlm}|dtd}nXt|dt}t|dt}|||j||d|d | d | }idd6}|r|j|_d|jkr|jd|dR$RP(R9((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pytresid_anscombe_scaleds cCs(|jj|j|jd|jddS(s Unscaled Anscombe residuals. See statsmodels.families.family for distribution-specific Anscombe residuals. RRPg?(RR=R.R>R$(R9((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pytresid_anscombe_unscaleds cCs.|jj|j|jd|jdd}|S(s} Deviance residuals. See statsmodels.families.family for distribution- specific deviance residuals. RRPg?(Rt resid_devR.R>R$(R9R((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pytresid_deviances  cCsN|j|jd|jj|j}||j|j9}tj|}|S(sx Pearson's Chi-Squared statistic is defined as the sum of the squares of the Pearson residuals. i(R.RRR^R%R0RR@(R9tchisqtchisqsum((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyt pearson_chi2s'cCs|jS(sY Linear predicted values for the fitted model. dot(exog, params) (R(R9((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyR>scCs|jj|jS(s$ See GLM docstring. (R/R\RU(R9((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyR#scCs|j}|j}tjt|df}|j}|jdt|drwt||d|j |j j St j ||d|j|j}|j j SdS(s1 Fitted values of the null model iRRRN(R.R/RRJRHRLRR7R RRR>RRR%R0(R9R.R/R:R;R((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyR'*s    cCs%|jj|j|j|j|jS(sk See statsmodels.families.family for the distribution-specific deviance functions. (RRR.RR$R#(R9((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyR>scCs%|jj|j|j|j|jS(scThe value of the deviance function for the model fit with a constant as the only regressor.(RRR.R'R$R#(R9((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyt null_devianceGsc Cs4|jj|j|jd|jd|jd|jS(sW Log-likelihood of the model fit with a constant as the only regressor RRRP(RROR.R'R$R#RP(R9((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pytllnullNs  c Cs|j}t|jtjrt|jjtjjr|jjjdkrtj|j |j d|j j }||j j:}n |j}|j|j |j d|jd|jd|}|S(s Value of the loglikelihood function evalued at params. See statsmodels.families.family for distribution-specific loglikelihoods. g?iRRRP(RR!RRDR"REtPowerRRR.RR%R@R/RARPROR$R#(R9t _modelfamilyRPR9((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyRXXs )    cCsd|jd|jdS(sV Akaike Information Criterion -2 * `llf` + 2*(`df_model` + 1) iii(RXR?(R9((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pytaicmscCs/|j|jj|jdtj|jjS(s[ Bayes Information Criterion `deviance` - `df_resid` * log(`nobs`) i(RR/RAR?RR)(R9((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pytbicusc Csddljj}i|d6|d6td6}|j|d|d|d|d|} t|dRR'RRFRGRXRJRKRmRlRQRRRSRWR^R_R tLikelihoodModelResultsRaRRkRRlRR|R}(((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyR]sT$?            #    B RcBsIeZidd6dd6dd6dd6dd6ZejejjeZRS(trowsR=RBR8R7R:(R(Rt_attrstwrapt union_dictsRtRegressionResultsWrappert _wrap_attrs(((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyRrs t__main__t as_pandastreturnsttablesRtsTest GLM(;R"tnumpyRRiRtstatsmodels.tools.decoratorsRtstatsmodels.base.modelR R/t#statsmodels.regression.linear_modelRPt linear_modelRtstatsmodels.base.wrappertwrapperRtstatsmodels.regression._toolst_toolsRt)statsmodels.graphics._regressionplots_docRRRRRRtstatsmodels.genmod._predictionRR5RRR tnumpy.linalg.linalgR t__all__RtLikelihoodModelR RRRRtpopulate_wrapperR(tstatsmodels.apitapitsmtdatasetstlongleytloadRltdataR.R:RtGLMmodR|tGLMTtGLMTp(((sJlib/python2.7/site-packages/statsmodels/genmod/generalized_linear_model.pyts@   .  #