B ZY@s8ddlZddlmZddlZddlmZmZddlZddl m Z ddl m Z m Z ddlmZmZmZddlmZdd Zd d Zd d ZddZddZddZddZddZGdddeZGdddeZedkr4ddlZddl m!Z!ej"ddd d!d"d#d$gd%Z#e!d&e#d'$Z%e!d(e#d'$Z&ee%d)d*Z'dS)+N)stats) DataFrameIndex)OLS)lrangelmap)_remove_intercept_patsy_has_intercept_intercept_idx)summary2cCsp|dkr|S|dkr$|j}|jS|dkr8|j}|jS|dkrL|j}|jS|dkr`|j}|jSt d|dS)NZhc0Zhc1Zhc2Zhc3z robust options %s not understood) Z cov_paramsZHC0_seZcov_HC0ZHC1_seZcov_HC1ZHC2_seZcov_HC2ZHC3_seZcov_HC3 ValueError)modelrobustZser6lib/python3.7/site-packages/statsmodels/stats/anova.py_get_covariance src Ks2|dd}|dd}|dd}|dd}|r<|}|jj}|jj}|jd}|jj} |jjj} |jj } t | j t | d} d |} d d d || g}t t| d f|d}|dkrt||||| || || | S|dkrt|| | || |S|dkr t|| | || |S|dkrtdntdt|dS)a9 Anova table for one fitted linear model. Parameters ---------- model : fitted linear model results instance A fitted linear model typ : int or str {1,2,3} or {"I","II","III"} Type of sum of squares to use. **kwargs** scale : float Estimate of variance, If None, will be estimated from the largest model. Default is None. test : str {"F", "Chisq", "Cp"} or None Test statistics to provide. Default is "F". Notes ----- Use of this function is discouraged. Use anova_lm instead. testFscaleNtyprrzPR(>%s)dfsum_sqmean_sq)columns)rI)ZII)ZIII)ZIVzType IV not yet implementedzType %s not understood)getlowerr endogexogshapeZ endog_namesdata design_info exog_nameslentermsr rnpzerosanova1_lm_singleanova2_lm_singleanova3_lm_singleNotImplementedr str)r kwargsrrrrr"r#nobsZ response_namer&r'n_rowspr_testnamestablerrr anova_single!s8            r7c st|dd} | dkr2tj|\} } t| j|} ttjtj f} fddj D}x t |D]\}}d| ||f<qjWt| | d}t }||}t j }||}|}t|dg|_tj| |d|f|j|dd gf<|j|jf|jdd dgf<|d krv|d |d|j|j||<tj|d |d|j||<tjtjf|jd||gf<|d |d|d <|S) a Anova table for one fitted linear model. Parameters ---------- model : fitted linear model results instance A fitted linear model **kwargs** scale : float Estimate of variance, If None, will be estimated from the largest model. Default is None. test : str {"F", "Chisq", "Cp"} or None Test statistics to provide. Default is "F". Notes ----- Use of this function is discouraged. Use anova_lm instead. effectsNcsg|]}|qSr)slice).0name)r&rr ysz$anova1_lm_single..rrResidualrrrr)getattrr*linalgqrdotTr+r(r)Z column_names term_names enumerater arraytolistrindexZc_sumlocssrdf_residrfsfnan)r r"r#r2r&r6r3rr4rr8qrZarrslicesiZslice_ridxrCrGr)r&rr,\s2    (  r,cCs|jdd}t|}dd||g}tt|df|d}t|d} t||} g} g} xt|D]\} }||}t|j |j }g}t |j }xZ|D]R}t |j }| |r||ks||}|t|j |j |t|j |j qWt|jjjd|}t|jjjd|}|jrtt|| |j}ddlm}||\}}|jd|jd}t|dd| dfj|}n|}|jd}|d kr|j|| d }|j|j|j| |f<}|j|j|j| |f<||j|j| df<| |j | |q`Wt | d g|_|j!t"| |jjjddg}|||d|j#|j$}||d<|j#|j$tj%tj%f|jd dd||gf<|S) a Anova type II table for one fitted linear model. Parameters ---------- model : fitted linear model results instance A fitted linear model **kwargs** scale : float Estimate of variance, If None, will be estimated from the largest model. Default is None. test : str {"F", "Chisq", "Cp"} or None Test statistics to provide. Default is "F". Notes ----- Use of this function is discouraged. Use anova_lm instead. Type II Sum of Squares compares marginal contribution of terms. Thus, it is not particularly useful for models with significant interaction terms. Nrrr)rrr)r?r)cov_pr=)&r)rrr*r+rrDr9rstartstopsetfactorsissubsetextendeyer r#r$sizerArBscipyr?r@f_testfvaluerIrGpvalueappendr;rilocZargsortrJrKrN)r r&r3rr4r terms_infor5r6covZ robust_cov col_orderrGrRtermcolsL1ZL2Zterm_settZ other_setcolZLVLr?Z orth_compl_rPL12rL test_valuerJrrrr-sX         "   $r-cCsR|t|7}|j}dd||g}tt|df|d}t||} g} g} xt|D]\} } || }t|j j j d|}|}|j d}|dkr|j || d}|j |j|j| |f<}|j|j|j| |f<||j|j| df<| | qPWt| d g|_|||d|j|j}||d<|j|jtjtjf|jd dd||gf<|S) Nrrr)rrrr)rTr=)r r)rr*r+rrDr9r[r r#r$r^r_rIrGr`rar;rrJrKrN)r r&r3rr4rrcr5r6rdrerGrRrfrgrhrlrPrLrmrJrrrr.s2     r.c Osv|dd}t|dkr,|d}t|f|Sy|dks:tWntdt|YnX|dd}|dd }t|}d |}d d d d||g}tt|df|d} |s|dj }t t |d g|| d <t t |d g|| d <t | d j  | j| jdd d f<| d  | d<|dkrr| d| d || d<tj| d| d | d | |<tj| || d<| S)a Anova table for one or more fitted linear models. Parameters ---------- args : fitted linear model results instance One or more fitted linear models scale : float Estimate of variance, If None, will be estimated from the largest model. Default is None. test : str {"F", "Chisq", "Cp"} or None Test statistics to provide. Default is "F". typ : str or int {"I","II","III"} or {1,2,3} The type of Anova test to perform. See notes. robust : {None, "hc0", "hc1", "hc2", "hc3"} Use heteroscedasticity-corrected coefficient covariance matrix. If robust covariance is desired, it is recommended to use `hc3`. Returns ------- anova : DataFrame A DataFrame containing. Notes ----- Model statistics are given in the order of args. Models must have been fit using the formula api. See Also -------- model_results.compare_f_test, model_results.compare_lm_test Examples -------- >>> import statsmodels.api as sm >>> from statsmodels.formula.api import ols >>> moore = sm.datasets.get_rdataset("Moore", "car", cache=True) # load >>> data = moore.data >>> data = data.rename(columns={"partner.status" : ... "partner_status"}) # make name pythonic >>> moore_lm = ols('conformity ~ C(fcategory, Sum)*C(partner_status, Sum)', ... data=data).fit() >>> table = sm.stats.anova_lm(moore_lm, typ=2) # Type 2 Anova DataFrame >>> print(table) rrr)rrz6Multiple models only supported for type I. Got type %srrrNzPr(>%s)rKrJZdf_diffZss_diff)r)r r(r7AssertionErrorr r0rr*r+rrr>ZdiffvaluesrIrGrrLrMrNZisnull) argsr1rr rrZn_modelsr4r5r6rrranova_lms6.      & rscCs2tdg|}x|D]}||}d||<qW|S)NTF)r*rE)rQZslices_to_excludenindrfsrrr _not_slicefs   rwc Cs\t|||jd}||}t||dd|f|}|j|}t|t|}||fS)ai Residual sum of squares of OLS model excluding factors in `keys` Assumes x matrix is orthogonal Parameters ---------- y : array_like dependent variable x : array_like independent variables term_slices : a dict of slices term_slices[key] is a boolean array specifies the parameters associated with the factor `key` params : ndarray OLS solution of y = x * params keys : keys for term_slices factors to be excluded Returns ------- rss : float residual sum of squares df : int degrees of freedom rN)rwr$r*subtractrArBr() yx term_slicesparamskeysruZparams1rJrKrrr_ssr_reduced_modelns  r~c@s2eZdZdZd ddZddZddZd d ZdS) AnovaRMa  Repeated measures Anova using least squares regression The full model regression residual sum of squares is used to compare with the reduced model for calculating the within-subject effect sum of squares [1]. Currently, only fully balanced within-subject designs are supported. Calculation of between-subject effects and corrections for violation of sphericity are not yet implemented. Parameters ---------- data : DataFrame depvar : string The dependent variable in `data` subject : string Specify the subject id within : a list of string(s) The within-subject factors between : a list of string(s) The between-subject factors, this is not yet implemented aggregate_func : None, 'mean', or function If the data set contains more than a single observation per subject and cell of the specified model, this function will be used to aggregate the data before running the Anova. `None` (the default) will not perform any aggregation; 'mean' is s shortcut to `numpy.mean`. An exception will be raised if aggregation is required, but no aggregation function was specified. Returns ------- results: AnovaResults instance Raises ------ ValueError If the data need to be aggregated, but `aggregate_func` was not specified. Notes ----- This implementation currently only supports fully balanced designs. If the data contain more than one observation per subject and cell of the design, these observations need to be aggregated into a single observation before the Anova is calculated, either manually or by passing an aggregation function via the `aggregate_func` keyword argument. Note that if the input data set was not balanced before performing the aggregation, the implied heteroscedasticity of the data is ignored. References ---------- .. [*] Rutherford, Andrew. Anova and ANCOVA: a GLM approach. John Wiley & Sons, 2011. NcCs||_||_||_d|kr"td||_|dk r8td||_|dkrPtj|_ n||_ | |j |g|ds|j dk r| n d}t|| dS)NCzSFactor name cannot be 'C'! This is in conflict with patsy's contrast function name.z)Between subject effect not yet supported!mean)ZsubsetzThe data set contains more than one observation per subject and cell. Either aggregate the data manually, or pass the `aggregate_func` parameter.)r%depvarwithinr betweenNotImplementedErrorsubjectr*raggregate_funcZequalsZdrop_duplicates _aggregate_check_data_balanced)selfr%rrrrrmsgrrr__init__s$   zAnovaRM.__init__cCs.|jj|jg|jdd|j|j|_dS)NF)Zas_index)r%groupbyrrrZaggr)rrrrrs zAnovaRM._aggregatec Csd}x$|jD]}|t|j|9}q Wi}xlt|jjdD]X}g}x$|jD]}||j|j|qPWt|}||kr||d||<q@d||<q@Wd}t||krt |||}x |D]}|||krt |qW|jjd||krt ddS)zraise if data is not balanced This raises a ValueError if the data is not balanced, and returns None if it is balance Return might change rrzData is unbalanced.z9There are more than 1 element in a cell! Missing factors?N) rr(r%uniqueranger$rarbtupler ) rZ factor_levelsZwiZ cell_countrGkeyrjZ error_messagecountrrrrs*        zAnovaRM._check_data_balancedcCs|j|jj}dd|jD}d|j}||g}tjd||jd}|jj }x<|D]4}t dg|j d}d|||<t |||<qVWd |g} t || |j d}|d d |f}t||} | } | j|j dkrtd x| D]} || qWx|D]}|||||<qW| j} | j}| j}tggggd }x|D]}|j|krL|d krLt|||| |g\}}||}|||}|d |d dks|d ||kr||}|}n2t|||| |d |g\}}||}|||}||}tj|||}|dddd}||j|df<||j|df<||j|df<||j|df<qLWt|jd d ddddgfS)zwestimate the model and compute the Anova table Returns ------- AnovaResults instance cSsg|] }d|qS)z C(%s, Sum)r)r:rRrrrr<szAnovaRM.fit..z C(%s, Sum)*)r%FrT:Nz$Independent variables are collinear.)zF ValuezNum DFzDen DFzPr > FZ InterceptrozC(z, Sum)zF ValuezNum DFzDen DFzPr > Frrr) r%rrqrrpatsyZdmatrixjoinr&Zterm_name_slicesr*rEr$rwrfitZrankr popr|rKrJpdrr~rrLrMreplacerI AnovaResultsrb)rryrrrXrzr{rruZ term_excluder ZresultsrRr|rKrJ anova_tableZssr1Z df_resid1Zdf1ZmsmZmseZdf2rprfrrrrsb          z AnovaRM.fit)NNN)__name__ __module__ __qualname____doc__rrrrrrrrrs 7 "rc@s(eZdZdZddZddZddZdS) rzX Anova results class Attributes ---------- anova_table : DataFrame cCs ||_dS)N)r)rrrrrr\szAnovaResults.__init__cCs |S)N)summary__str__)rrrrr_szAnovaResults.__str__cCs"t}|d||j|S)zdcreate summary results Returns ------- summary : Summary instance ZAnova)r ZSummaryZ add_titleZadd_dfr)rZsummrrrrbs  zAnovaResults.summaryN)rrrrrrrrrrrrTsr__main__)olsz moore.csv,rZpartner_statusZ conformityZ fcategoryZfscore)Z delimiterZskiprowsr5z5conformity ~ C(fcategory, Sum)*C(partner_status, Sum))r%z#conformity ~ C(partner_status, Sum)r)r)(Znumpyr*r]rZpandasrrrrZ#statsmodels.regression.linear_modelrZstatsmodels.compat.pythonrrZ statsmodels.formula.formulatoolsrr r Zstatsmodels.iolibr rr7r,r-r.rsrwr~objectrrrZstatsmodels.formula.apirZ read_tableZmoorerZmoore_lmZmooreBr6rrrrs:   ;7X'T#D