ó áp7]c @s,ddlmZmZddlZddlmZddlmZddl m Z ddl m Z ddl mZdd lmZd d d d gZd efd„ƒYZejfdddeeed„Zeeeed„Zeeedd„Zd„Zd„Zeeedd„Zd„ZdS(iÿÿÿÿ(tlzipt string_typesN(tstats(tOLS(t add_constant(tcache_readonly(tECDFi(tutilstqqplottqqplot_2samplestqqlinetProbPlotcBs¿eZdZejed dddd„Zed„ƒZed„ƒZ ed„ƒZ ed„ƒZ ed„ƒZ d d d d d d „Zd d d d d d „Zd d d ed d „ZRS(sA Class for convenient construction of Q-Q, P-P, and probability plots. Can take arguments specifying the parameters for dist or fit them automatically. (See fit under kwargs.) Parameters ---------- data : array-like 1d data array dist : A scipy.stats or statsmodels distribution Compare x against dist. The default is scipy.stats.distributions.norm (a standard normal). distargs : tuple A tuple of arguments passed to dist to specify it fully so dist.ppf may be called. distargs must not contain loc or scale. These values must be passed using the loc or scale inputs. a : float Offset for the plotting position of an expected order statistic, for example. The plotting positions are given by (i - a)/(nobs - 2*a + 1) for i in range(0,nobs+1) loc : float Location parameter for dist scale : float Scale parameter for dist fit : boolean If fit is false, loc, scale, and distargs are passed to the distribution. If fit is True then the parameters for dist are fit automatically using dist.fit. The quantiles are formed from the standardized data, after subtracting the fitted loc and dividing by the fitted scale. See Also -------- scipy.stats.probplot Notes ----- 1) Depends on matplotlib. 2) If `fit` is True then the parameters are fit using the distribution's `fit()` method. 3) The call signatures for the `qqplot`, `ppplot`, and `probplot` methods are similar, so examples 1 through 4 apply to all three methods. 4) The three plotting methods are summarized below: ppplot : Probability-Probability plot Compares the sample and theoretical probabilities (percentiles). qqplot : Quantile-Quantile plot Compares the sample and theoretical quantiles probplot : Probability plot Same as a Q-Q plot, however probabilities are shown in the scale of the theoretical distribution (x-axis) and the y-axis contains unscaled quantiles of the sample data. Examples -------- The first example shows a Q-Q plot for regression residuals >>> # example 1 >>> import statsmodels.api as sm >>> from matplotlib import pyplot as plt >>> data = sm.datasets.longley.load(as_pandas=False) >>> data.exog = sm.add_constant(data.exog) >>> model = sm.OLS(data.endog, data.exog) >>> mod_fit = model.fit() >>> res = mod_fit.resid # residuals >>> probplot = sm.ProbPlot(res) >>> fig = probplot.qqplot() >>> h = plt.title('Ex. 1 - qqplot - residuals of OLS fit') >>> plt.show() qqplot of the residuals against quantiles of t-distribution with 4 degrees of freedom: >>> # example 2 >>> import scipy.stats as stats >>> probplot = sm.ProbPlot(res, stats.t, distargs=(4,)) >>> fig = probplot.qqplot() >>> h = plt.title('Ex. 2 - qqplot - residuals against quantiles of t-dist') >>> plt.show() qqplot against same as above, but with mean 3 and std 10: >>> # example 3 >>> probplot = sm.ProbPlot(res, stats.t, distargs=(4,), loc=3, scale=10) >>> fig = probplot.qqplot() >>> h = plt.title('Ex. 3 - qqplot - resids vs quantiles of t-dist') >>> plt.show() Automatically determine parameters for t distribution including the loc and scale: >>> # example 4 >>> probplot = sm.ProbPlot(res, stats.t, fit=True) >>> fig = probplot.qqplot(line='45') >>> h = plt.title('Ex. 4 - qqplot - resids vs. quantiles of fitted t-dist') >>> plt.show() A second `ProbPlot` object can be used to compare two separate sample sets by using the `other` kwarg in the `qqplot` and `ppplot` methods. >>> # example 5 >>> import numpy as np >>> x = np.random.normal(loc=8.25, scale=2.75, size=37) >>> y = np.random.normal(loc=8.75, scale=3.25, size=37) >>> pp_x = sm.ProbPlot(x, fit=True) >>> pp_y = sm.ProbPlot(y, fit=True) >>> fig = pp_x.qqplot(line='45', other=pp_y) >>> h = plt.title('Ex. 5 - qqplot - compare two sample sets') >>> plt.show() In qqplot, sample size of `other` can be equal or larger than the first. In case of larger, size of `other` samples will be reduced to match the size of the first by interpolation >>> # example 6 >>> x = np.random.normal(loc=8.25, scale=2.75, size=37) >>> y = np.random.normal(loc=8.75, scale=3.25, size=57) >>> pp_x = sm.ProbPlot(x, fit=True) >>> pp_y = sm.ProbPlot(y, fit=True) >>> fig = pp_x.qqplot(line='45', other=pp_y) >>> title = 'Ex. 6 - qqplot - compare different sample sizes' >>> h = plt.title(title) >>> plt.show() In ppplot, sample size of `other` and the first can be different. `other` will be used to estimate an empirical cumulative distribution function (ECDF). ECDF(x) will be plotted against p(x)=0.5/n, 1.5/n, ..., (n-0.5)/n where x are sorted samples from the first. >>> # example 7 >>> x = np.random.normal(loc=8.25, scale=2.75, size=37) >>> y = np.random.normal(loc=8.75, scale=3.25, size=57) >>> pp_x = sm.ProbPlot(x, fit=True) >>> pp_y = sm.ProbPlot(y, fit=True) >>> fig = pp_y.ppplot(line='45', other=pp_x) >>> h = plt.title('Ex. 7A- ppplot - compare two sample sets, other=pp_x') >>> fig = pp_x.ppplot(line='45', other=pp_y) >>> h = plt.title('Ex. 7B- ppplot - compare two sample sets, other=pp_y') >>> plt.show() The following plot displays some options, follow the link to see the code. .. plot:: plots/graphics_gofplots_qqplot.py iic Csà||_||_|jd|_||_||_t|tƒrUtt |ƒ}n|j|ƒ|_ |rè|j d|_ |j d|_ t |j ƒdkrÍ||j d tddddƒŽ|_qÓ|ddddƒ|_në|s|dks|dkr¸y%||td|d|ƒŽ|_Wnutk r¢djg|D]}t|ƒ^qHƒ}d } | jd |d|d|ƒ} td jd | ƒƒ‚nX||_ ||_ n||_||_ ||_ i|_dS( Niiþÿÿÿiÿÿÿÿitloctscaleis, s*dist({distargs}, loc={loc}, scale={scale})tdistargssŠInitializing the distribution failed. This can occur if distargs contains loc or scale. The distribution initialization command is: {cmd}tcmd(tdatatatshapetnobsRtfitt isinstanceRtgetattrRt fit_paramsR R tlentdicttdistt Exceptiontjointstrtformatt TypeErrort_cache( tselfRRRRRR R tdaR((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyt__init__¤s<     % (      cCst|j|jƒS(sTheoretical percentiles(t plotting_posRR(R!((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyttheoretical_percentilesÎscCsny|jj|jƒSWnPtk rKdj|jjƒ}t|ƒ‚ndj|jjƒ}‚nXdS(sTheoretical quantiless*%s requires more parameters to compute ppfs failed to compute the ppf of {0}N(RtppfR%RRtname(R!tmsg((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyttheoretical_quantilesÓs cCs&tj|jdtƒ}|jƒ|S(s sorted datatcopy(tnptarrayRtTruetsort(R!t sorted_data((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyR/às cCsG|jr<|jdkr<|jdkr<|j|j|jS|jSdS(ssample quantilesiiN(RR R R/(R!((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pytsample_quantilesçs'cCs/|j|jd|jd}|jj|ƒS(sSample percentilesiþÿÿÿiÿÿÿÿ(R/RRtcdf(R!t quantiles((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pytsample_percentilesïsc KsF|d k r¨t|tƒ}|s0t|ƒ}n|j}t|jƒ|jƒ} t|| |jd|d||\} }|d krd}n|d krd}qnZt|j|j|jd|d||\} }|d kríd}n|d krd}n|j |ƒ|j |ƒ|j ddgƒ|j ddgƒ| S( s8 P-P plot of the percentiles (probabilities) of x versus the probabilities (percetiles) of a distribution. Parameters ---------- xlabel : str or None, optional User-provided lables for the x-axis. If None (default), other values are used depending on the status of the kwarg `other`. ylabel : str or None, optional User-provided lables for the y-axis. If None (default), other values are used depending on the status of the kwarg `other`. line : str {'45', 's', 'r', q'} or None, optional Options for the reference line to which the data is compared: - '45': 45-degree line - 's': standardized line, the expected order statistics are scaled by the standard deviation of the given sample and have the mean added to them - 'r': A regression line is fit - 'q': A line is fit through the quartiles. - None: by default no reference line is added to the plot. other : ProbPlot, array-like, or None, optional If provided, ECDF(x) will be plotted against p(x) where x are sorted samples from `self`. ECDF is an empirical cumulative distribution function estimated from `other` and p(x) = 0.5/n, 1.5/n, ..., (n-0.5)/n where n is the number of samples in `self`. If an array-object is provided, it will be turned into a `ProbPlot` instance default parameters. If not provided (default), `self.dist(x)` is be plotted against p(x). ax : Matplotlib AxesSubplot instance, optional If given, this subplot is used to plot in instead of a new figure being created. **plotkwargs : additional matplotlib arguments to be passed to the `plot` command. Returns ------- fig : Matplotlib figure instance If `ax` is None, the created figure. Otherwise the figure to which `ax` is connected. taxtlinesProbabilities of 2nd SamplesProbabilities of 1st SamplesTheoretical ProbabilitiessSample Probabilitiesggð?N( tNoneRR R%RR0t_do_plotRR3t set_xlabelt set_ylabeltset_xlimtset_ylim( R!txlabeltylabelR5totherR4t plotkwargst check_othertp_xtecdf_xtfig((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pytppplotös2.             c Ks|d k r t|tƒ}|s0t|ƒ}n|j}|j} t|ƒt| ƒkrmtddƒ‚nEt|ƒt| ƒkr²t|j|jƒ} t j j | | ƒ} nt | ||j d|d||\} }|d krñd}n|d krcd}qcnZt |j|j|j d|d||\} }|d krNd}n|d krcd}n|j|ƒ|j|ƒ| S( s Q-Q plot of the quantiles of x versus the quantiles/ppf of a distribution or the quantiles of another `ProbPlot` instance. Parameters ---------- xlabel, ylabel : str or None, optional User-provided lables for the x-axis and y-axis. If None (default), other values are used depending on the status of the kwarg `other`. line : str {'45', 's', 'r', q'} or None, optional Options for the reference line to which the data is compared: - '45' - 45-degree line - 's' - standardized line, the expected order statistics are scaled by the standard deviation of the given sample and have the mean added to them - 'r' - A regression line is fit - 'q' - A line is fit through the quartiles. - None - by default no reference line is added to the plot. other : `ProbPlot` instance, array-like, or None, optional If provided, the sample quantiles of this `ProbPlot` instance are plotted against the sample quantiles of the `other` `ProbPlot` instance. Sample size of `other` must be equal or larger than this `ProbPlot` instance. If the sample size is larger, sample quantiles of `other` will be interpolated to match the sample size of this `ProbPlot` instance. If an array-like object is provided, it will be turned into a `ProbPlot` instance using default parameters. If not provided (default), the theoretical quantiles are used. ax : Matplotlib AxesSubplot instance, optional If given, this subplot is used to plot in instead of a new figure being created. **plotkwargs : additional matplotlib arguments to be passed to the `plot` command. Returns ------- fig : Matplotlib figure instance If `ax` is None, the created figure. Otherwise the figure to which `ax` is connected. s(Sample size of `other` must be equal or s$larger than this `ProbPlot` instanceR4R5sQuantiles of 2nd SamplesQuantiles of 1st SamplesTheoretical QuantilessSample QuantilesN(R6RR R0Rt ValueErrorR$RRRtmstatst mquantilesR7RR)R8R9( R!R<R=R5R>R4R?R@ts_selfts_othertpRC((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyRFs:,               c Ksé|r[t|jddd…|j|jd|d||\}}|dkr d}q nEt|j|j|jd|d||\}}|dkr d}n|dkrµd}n|j|ƒ|j|ƒt||j|jƒ|S(su Probability plot of the unscaled quantiles of x versus the probabilities of a distibution (not to be confused with a P-P plot). The x-axis is scaled linearly with the quantiles, but the probabilities are used to label the axis. Parameters ---------- xlabel, ylabel : str or None, optional User-provided lables for the x-axis and y-axis. If None (default), other values are used depending on the status of the kwarg `other`. line : str {'45', 's', 'r', q'} or None, optional Options for the reference line to which the data is compared: - '45' - 45-degree line - 's' - standardized line, the expected order statistics are scaled by the standard deviation of the given sample and have the mean added to them - 'r' - A regression line is fit - 'q' - A line is fit through the quartiles. - None - by default no reference line is added to the plot. exceed : boolean, optional - If False (default) the raw sample quantiles are plotted against the theoretical quantiles, show the probability that a sample will not exceed a given value - If True, the theoretical quantiles are flipped such that the figure displays the probability that a sample will exceed a given value. ax : Matplotlib AxesSubplot instance, optional If given, this subplot is used to plot in instead of a new figure being created. **plotkwargs : additional matplotlib arguments to be passed to the `plot` command. Returns ------- fig : Matplotlib figure instance If `ax` is None, the created figure. Otherwise the figure to which `ax` is connected. NiÿÿÿÿR4R5sProbability of Exceedance (%)sNon-exceedance Probability (%)sSample Quantiles( R7R)R/RR6R8R9t_fmt_probplot_axisR(R!R<R=R5texceedR4R?RC((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pytprobplotšs&-         (N(t__name__t __module__t__doc__RtnormtFalseR#RR%R)R/R0R3R6RDRRM(((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyR s“ )   O  S ic KsOt|d|d|d|d|d|d|ƒ} | jd|d|| } | S( sÞ Q-Q plot of the quantiles of x versus the quantiles/ppf of a distribution. Can take arguments specifying the parameters for dist or fit them automatically. (See fit under Parameters.) Parameters ---------- data : array-like 1d data array dist : A scipy.stats or statsmodels distribution Compare x against dist. The default is scipy.stats.distributions.norm (a standard normal). distargs : tuple A tuple of arguments passed to dist to specify it fully so dist.ppf may be called. loc : float Location parameter for dist a : float Offset for the plotting position of an expected order statistic, for example. The plotting positions are given by (i - a)/(nobs - 2*a + 1) for i in range(0,nobs+1) scale : float Scale parameter for dist fit : boolean If fit is false, loc, scale, and distargs are passed to the distribution. If fit is True then the parameters for dist are fit automatically using dist.fit. The quantiles are formed from the standardized data, after subtracting the fitted loc and dividing by the fitted scale. line : str {'45', 's', 'r', q'} or None Options for the reference line to which the data is compared: - '45' - 45-degree line - 's' - standardized line, the expected order statistics are scaled by the standard deviation of the given sample and have the mean added to them - 'r' - A regression line is fit - 'q' - A line is fit through the quartiles. - None - by default no reference line is added to the plot. ax : Matplotlib AxesSubplot instance, optional If given, this subplot is used to plot in instead of a new figure being created. **plotkwargs : additional matplotlib arguments to be passed to the `plot` command. Returns ------- fig : Matplotlib figure instance If `ax` is None, the created figure. Otherwise the figure to which `ax` is connected. See Also -------- scipy.stats.probplot Examples -------- >>> import statsmodels.api as sm >>> from matplotlib import pyplot as plt >>> data = sm.datasets.longley.load(as_pandas=False) >>> data.exog = sm.add_constant(data.exog) >>> mod_fit = sm.OLS(data.endog, data.exog).fit() >>> res = mod_fit.resid # residuals >>> fig = sm.qqplot(res) >>> plt.show() qqplot of the residuals against quantiles of t-distribution with 4 degrees of freedom: >>> import scipy.stats as stats >>> fig = sm.qqplot(res, stats.t, distargs=(4,)) >>> plt.show() qqplot against same as above, but with mean 3 and std 10: >>> fig = sm.qqplot(res, stats.t, distargs=(4,), loc=3, scale=10) >>> plt.show() Automatically determine parameters for t distribution including the loc and scale: >>> fig = sm.qqplot(res, stats.t, fit=True, line='45') >>> plt.show() The following plot displays some options, follow the link to see the code. .. plot:: plots/graphics_gofplots_qqplot.py Notes ----- Depends on matplotlib. If `fit` is True then the parameters are fit using the distribution's fit() method. RRRRR R R4R5(R R( RRRRR R RR5R4R?RMRC((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyRásbc Csjt|tƒst|ƒ}nt|tƒs<t|ƒ}n|jd|d|d|d|d|ƒ}|S(s! Q-Q Plot of two samples' quantiles. Can take either two `ProbPlot` instances or two array-like objects. In the case of the latter, both inputs will be converted to `ProbPlot` instances using only the default values - so use `ProbPlot` instances if finer-grained control of the quantile computations is required. Parameters ---------- data1, data2 : array-like (1d) or `ProbPlot` instances xlabel, ylabel : str or None User-provided labels for the x-axis and y-axis. If None (default), other values are used. line : str {'45', 's', 'r', q'} or None Options for the reference line to which the data is compared: - '45' - 45-degree line - 's' - standardized line, the expected order statistics are scaled by the standard deviation of the given sample and have the mean added to them - 'r' - A regression line is fit - 'q' - A line is fit through the quartiles. - None - by default no reference line is added to the plot. ax : Matplotlib AxesSubplot instance, optional If given, this subplot is used to plot in instead of a new figure being created. Returns ------- fig : Matplotlib figure instance If `ax` is None, the created figure. Otherwise the figure to which `ax` is connected. See Also -------- scipy.stats.probplot Examples -------- >>> import statsmodels.api as sm >>> import numpy as np >>> import matplotlib.pyplot as plt >>> from statsmodels.graphics.gofplots import qqplot_2samples >>> x = np.random.normal(loc=8.5, scale=2.5, size=37) >>> y = np.random.normal(loc=8.0, scale=3.0, size=37) >>> pp_x = sm.ProbPlot(x) >>> pp_y = sm.ProbPlot(y) >>> qqplot_2samples(pp_x, pp_y) >>> plt.show() .. plot:: plots/graphics_gofplots_qqplot_2samples.py >>> fig = qqplot_2samples(pp_x, pp_y, xlabel=None, ylabel=None, ... line=None, ax=None) Notes ----- 1) Depends on matplotlib. 2) If `data1` and `data2` are not `ProbPlot` instances, instances will be created using the default parameters. Therefore, it is recommended to use `ProbPlot` instance if fine-grained control is needed in the computation of the quantiles. R<R=R5R>R4(RR R(tdata1tdata2R<R=R5R4RC((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyR IsDsr-c CsÊ|dkr€t|jƒ|jƒƒ}t|dƒ|d>> import statsmodels.api as sm >>> import numpy as np >>> import matplotlib.pyplot as plt >>> from statsmodels.graphics.gofplots import qqline >>> foodexp = sm.datasets.engel.load(as_pandas=False) >>> x = foodexp.exog >>> y = foodexp.endog >>> ax = plt.subplot(111) >>> plt.scatter(x, y) >>> ax.set_xlabel(foodexp.exog_name[0]) >>> ax.set_ylabel(foodexp.endog_name) >>> qqline(ax, 'r', x, y) >>> plt.show() .. plot:: plots/graphics_gofplots_qqplot_qqline.py t45iiNs*If line is not 45, x and y cannot be None.trtstqiiKgÐ?gè?(Rtget_xlimtget_ylimtmintmaxtplotR:R;R6RERRRt fittedvalueststdtmeant_check_for_ppfRtscoreatpercentileR&R+tdiff( R4R5txtyRtfmttend_ptstmtbtref_linetq25tq75ttheoretical_quartiles((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyR ™s26       cCs(tjd|dƒ||d|dS(s Generates sequence of plotting positions Parameters ---------- nobs : int Number of probability points to plot a : float Offset for the plotting position of an expected order statistic, for example. Returns ------- plotting_positions : array The plotting positions Notes ----- The plotting positions are given by (i - a)/(nobs - 2*a + 1) for i in range(0,nobs+1) See Also -------- scipy.stats.mstats.plotting_positions gð?ii(R+tarange(RR((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyR$íscCs†t|ƒ|dkrYtjddddddddd d d d d ddgƒd}nÅ|dkrºtjdddddddddddd d d d d dddddgƒd}ndtjddddddddddddddd d d d d ddddddddgƒd}|j|ƒ}|j|ƒ|j|ddd d!d"d#d$d%d&ƒ|j|jƒ|jƒgƒd'S((s Formats a theoretical quantile axis to display the corresponding probabilities on the quantiles' scale. Parameteters ------------ ax : Matplotlib AxesSubplot instance, optional The axis to be formatted nobs : scalar Numbero of observations in the sample dist : scipy.stats.distribution A scipy.stats distribution sufficiently specified to impletment its ppf() method. Returns ------- There is no return value. This operates on `ax` in place i2iiii iii(i<iFiPiZi_ibicgY@iôgš™™™™™¹?gš™™™™™É?gà?gàX@g33333óX@gš™™™™ùX@g{®Gáz„?g{®Gáz”?gš™™™™™©?gÍÌÌÌÌüX@g…ëQ¸þX@gÂõ(\ÿX@idtrotationi-t rotation_modetanchorthorizontalalignmenttrighttverticalalignmenttcenterN( RaR+R,R&t set_xtickstset_xticklabelsR:R[R\(R4RRt axis_probst axis_qntls((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyRK s$  !" *$! tboc Kstj|ƒ\}}|jdƒ|j|||||r…|d krcd|}t|ƒ‚nt||d|d|d |ƒn||fS( sJ Boiler plate plotting function for the `ppplot`, `qqplot`, and `probplot` methods of the `ProbPlot` class Parameteters ------------ x, y : array-like Data to be plotted dist : scipy.stats.distribution A scipy.stats distribution, needed if `line` is 'q'. line : str {'45', 's', 'r', q'} or None Options for the reference line to which the data is compared. ax : Matplotlib AxesSubplot instance, optional If given, this subplot is used to plot in instead of a new figure being created. fmt : str, optional matplotlib-compatible formatting string for the data markers kwargs : keywords These are passed to matplotlib.plot Returns ------- fig : Matplotlib Figure instance ax : Matplotlib AxesSubplot instance (see Parameters) g{®Gáz”?RVRXRURWs!%s option for line not understoodRdReR(RVRXRURW(Rt create_mpl_axt set_xmarginR]RER ( RdReRR5R4RftkwargsRCR(((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyR72s   "cCs"t|dƒstdƒ‚ndS(NR&s#distribution must have a ppf method(thasattrRE(R((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyRaZs(tstatsmodels.compat.pythonRRtnumpyR+tscipyRt#statsmodels.regression.linear_modelRtstatsmodels.tools.toolsRtstatsmodels.tools.decoratorsRtstatsmodels.distributionsRtRt__all__tobjectR RQRRR6RR R R$RKR7Ra(((s<lib/python2.7/site-packages/statsmodels/graphics/gofplots.pyts& ÿÓg  OT  ((