ó áp7]c@ s›dZddlmZmZddlZddlZddlm Z ddl m Z m Z d„Z defd„ƒYZeeeeeed d „ZdS( sLPrincipal Component Analysis Author: josef-pktd Modified by Kevin Sheppard iÿÿÿÿ(tprint_functiontdivisionN(trange(t ValueWarningtEstimationWarningcC stjtj||ƒƒS(N(tnptsqrttsum(tx((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyt_normstPCAc B sïeZdZdeeeedddddddd„ Zd„Zd„Zd„Z d „Z d „Z d „Z d „Z d „Zd„Zd„Zd„Zd„Zd„Zdeed„Zd„Zdeedd„Zddd„ZRS(s> Principal Component Analysis Parameters ---------- data : array-like Variables in columns, observations in rows ncomp : int, optional Number of components to return. If None, returns the as many as the smaller of the number of rows or columns in data standardize: bool, optional Flag indicating to use standardized data with mean 0 and unit variance. standardized being True implies demean. Using standardized data is equivalent to computing principal components from the correlation matrix of data demean : bool, optional Flag indicating whether to demean data before computing principal components. demean is ignored if standardize is True. Demeaning data but not standardizing is equivalent to computing principal components from the covariance matrix of data normalize : bool , optional Indicates whether th normalize the factors to have unit inner product. If False, the loadings will have unit inner product. weights : array, optional Series weights to use after transforming data according to standardize or demean when computing the principal components. gls : bool, optional Flag indicating to implement a two-step GLS estimator where in the first step principal components are used to estimate residuals, and then the inverse residual variance is used as a set of weights to estimate the final principal components. Setting gls to True requires ncomp to be less then the min of the number of rows or columns method : str, optional Sets the linear algebra routine used to compute eigenvectors 'svd' uses a singular value decomposition (default). 'eig' uses an eigenvalue decomposition of a quadratic form 'nipals' uses the NIPALS algorithm and can be faster than SVD when ncomp is small and nvars is large. See notes about additional changes when using NIPALS tol : float, optional Tolerance to use when checking for convergence when using NIPALS max_iter : int, optional Maximum iterations when using NIPALS missing : string Method for missing data. Choices are 'drop-row' - drop rows with missing values 'drop-col' - drop columns with missing values 'drop-min' - drop either rows or columns, choosing by data retention 'fill-em' - use EM algorithm to fill missing value. ncomp should be set to the number of factors required tol_em : float Tolerance to use when checking for convergence of the EM algorithm max_em_iter : int Maximum iterations for the EM algorithm Attributes ---------- factors : array or DataFrame nobs by ncomp array of of principal components (scores) scores : array or DataFrame nobs by ncomp array of of principal components - identical to factors loadings : array or DataFrame ncomp by nvar array of principal component loadings for constructing the factors coeff : array or DataFrame nvar by ncomp array of principal component loadings for constructing the projections projection : array or DataFrame nobs by var array containing the projection of the data onto the ncomp estimated factors rsquare : array or Series ncomp array where the element in the ith position is the R-square of including the fist i principal components. Note: values are calculated on the transformed data, not the original data ic : array or DataFrame ncomp by 3 array containing the Bai and Ng (2003) Information criteria. Each column is a different criteria, and each row represents the number of included factors. eigenvals : array or Series nvar array of eigenvalues eigenvecs : array or DataFrame nvar by nvar array of eigenvectors weights : array nvar array of weights used to compute the principal components, normalized to unit length transformed_data : array Standardized, demeaned and weighted data used to compute principal components and related quantities cols : array Array of indices indicating columns used in the PCA rows : array Array of indices indicating rows used in the PCA Examples -------- Basic PCA using the correlation matrix of the data >>> import numpy as np >>> from statsmodels.multivariate.pca import PCA >>> x = np.random.randn(100)[:, None] >>> x = x + np.random.randn(100, 100) >>> pc = PCA(x) Note that the principal components are computed using a SVD and so the correlation matrix is never constructed, unless method='eig'. PCA using the covariance matrix of the data >>> pc = PCA(x, standardize=False) Limiting the number of factors returned to 1 computed using NIPALS >>> pc = PCA(x, ncomp=1, method='nipals') >>> pc.factors.shape (100, 1) Notes ----- The default options perform principal component analysis on the demeaned, unit variance version of data. Setting standardize to False will instead only demean, and setting both standardized and demean to False will not alter the data. Once the data have been transformed, the following relationships hold when the number of components (ncomp) is the same as tne minimum of the number of observation or the number of variables. .. math: X' X = V \Lambda V' .. math: F = X V .. math: X = F V' where X is the `data`, F is the array of principal components (`factors` or `scores`), and V is the array of eigenvectors (`loadings`) and V' is the array of factor coefficients (`coeff`). When weights are provided, the principal components are computed from the modified data .. math: \Omega^{-\frac{1}{2}} X where :math:`\Omega` is a diagonal matrix composed of the weights. For example, when using the GLS version of PCA, the elements of :math:`\Omega` will be the inverse of the variances of the residuals from .. math: X - F V' where the number of factors is less than the rank of X .. [*] J. Bai and S. Ng, "Determining the number of factors in approximate factor models," Econometrica, vol. 70, number 1, pp. 191-221, 2002 tsvdgH¯¼šò×j>ièidcC sÐd|_g|_t|tjƒr?|j|_|j|_ntj |ƒ|_ ||_ ||_ | |_ d|j ko†dknsštdƒ‚n| |_| |_| |_||_||_|j j\|_|_|dkrtj|jƒ}nWtj|ƒjƒ}|jd|jkr:tdƒ‚n|tj|djƒƒ}||_t|j|jƒ}|dkr‡|n||_|j|krÍddl}d}|j |t!ƒ||_n||_"|j"dkrýtd j#|ƒƒ‚ntj$|jƒ|_%tj$|jƒ|_&| |_'|j |_(| dk rÓ|j)ƒ|j(j\|_|_|jtj|j jƒkr£tj|j(jƒ|_qÓ|jtj|j(jƒkrÓtd ƒ‚qÓnd |_*d|_+d|_,d|_-d|_.d|_/d|_0d|_1|_2d|_3d|_4d|_5d|_6d|_7d|_8d|_9|j:ƒ|_,|j;ƒ|r¦|j<ƒ|j:ƒ|_,|j;ƒn|j=ƒ|jdk rÌ|j>ƒndS(Niis$tol must be strictly between 0 and 1s!weights should have nvar elementsg@iÿÿÿÿs¥The requested number of components is more than can be computed from data. The maximum number of components is the minimum of the number of observations or variablesteigR tnipalssmethod {0} is not known.sWhen adjusting for missing values, user provided ncomp must be no larger than the smallest dimension of the missing-value-adjusted data size.g(R R R (?tNonet_indext_columnst isinstancetpdt DataFrametindextcolumnsRtasarraytdatat_glst _normalizet_tolt ValueErrort _max_itert _max_em_itert_tol_emt _standardizet_demeantshapet_nobst_nvartonestarraytflattenRtmeantweightstmint_ncomptwarningstwarnRt_methodtformattarangetrowstcolst_missingt_adjusted_datat_adjust_missingt_tsst_essttransformed_datat_mut_sigmat _ess_indivt _tss_indivtscorestfactorstloadingstcoefft eigenvalst eigenvecst projectiontrsquaretict _prepare_datat_pcat_compute_gls_weightst_compute_rsquare_and_ict _to_pandas(tselfRtncompt standardizetdemeant normalizetglsR(tmethodtmissingttoltmax_iterttol_emt max_em_itertmin_dimR+R,((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyt__init__¹sŠ                                      c C s×d„}d„}|jdkrb||jƒ\|_}tj|ƒd|_|j||_n|jdkr¢||jƒ\|_}tj|ƒd|_nØ|jdkrM||jƒ\}}|j}||jƒ\}}|j} | |kr||_tj|ƒd|_qz||_|j||_tj|ƒd|_n-|jdkrn|j ƒ|_n t dƒ‚|j d k r²|j |j|_ |j |j|_ n|jjdkrÓt d ƒ‚nd S( sE Implements alternatives for handling missing values cS s>tjtjtj|ƒdƒƒ}|dd…|f|fS(Ni(Rt logical_nottanytisnan(RR((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pytkeep_col"s$cS s>tjtjtj|ƒdƒƒ}||dd…f|fS(Ni(RRXRYRZ(RR((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pytkeep_row&s$sdrop-colisdrop-rowsdrop-minsfill-emsmissing method is not known.s2Removal of missing values has eliminated all data.N(R2RR3RtwhereR1R(R0tsizet_fill_missing_emRRRR( RJR[R\Rtdrop_coltdrop_col_indext drop_col_sizetdrop_rowtdrop_row_indext drop_row_size((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyR4s8        c C s|jtj|jƒ}|j|jkr:tdƒ‚n|djdƒ}d|}|tj|djƒƒ}|j}dt ||j ƒdƒ|}|dkrût tj ||ƒƒ}ddl }dj d |d |ƒ}|j|tƒn||_dS( sF Computes GLS weights based on percentage of data fit sOgls can only be used when ncomp < nvar so that residuals have non-zero varianceg@igð?gš™™™™™¹?iÿÿÿÿNs‰Many series are being down weighted by GLS. Of the {original} series, the GLS estimates are based on only {effective} (effective) series.toriginalt effective(R7RRRBR*R#RR'RRtinttroundR+R.R,RR(( RJterrorstvarR(tnvarteff_series_perct eff_seriesR+R,((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyRGMs  "   cC s'|jƒ|jƒ|jƒ|_dS(s" Main PCA routine N(t _compute_eigt_compute_pca_from_eigtprojectRB(RJ((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyRFfs  cC s8|jƒ}|d }|dtt|ƒƒd7}|S(Niÿÿÿÿs, id: t)(t__str__thextid(RJtstring((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyt__repr__ns  cC sñd}|dt|jƒd7}|dt|jƒd7}|jrNd}n|jr`d}nd}|d|d7}|jrŽ|d 7}n|d t|jƒd7}|d t|jƒd7}||jd krÜdnd7}|d7}|S(NsPrincipal Component Analysis(snobs: s, snvar: sStandardize (Correlation)sDemean (Covariance)Rstransformation: sGLS, snormalization: snumber of components: R smethod: t EigenvaluetSVDRrsmethod: Eigenvalue( tstrR"R#RR RRR*R-(RJRvtkind((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyRsts        cC sØ|j}tjtj|ƒƒrAtj|jdƒjtjƒStj|ddƒ|_ tj tj||j dddƒƒ|_ |j r¥||j |j }n|j r¾||j }n|}|tj |jƒS(s- Standardize or demean data. itaxisig@(R3RtallRZtemptyR!tfilltnantnanmeanR8RR9RR R((RJtadj_dataR((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyRE‡s  ,  cC s@|jdkr|jƒS|jdkr2|jƒS|jƒSdS(sz Wrapper for actual eigenvalue method This is a workaround to avoid instance methods in __dict__ R R N(R-t_compute_using_eigt_compute_using_svdt_compute_using_nipals(RJ((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyRo™s   cC sA|j}tjj|ƒ\}}}|d|_|j|_dS(s/SVD method to compute eigenvalues and eigenvecsg@N(R7RtlinalgR R@tTRA(RJRtutstv((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyR„¦s  cC s7|j}tjj|jj|ƒƒ\|_|_dS(sY Eigenvalue decomposition method to compute eigenvalues and eigenvectors N(R7RR†teighR‡tdotR@RA(RJR((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyRƒ­s cC sÔ|j}|jdkr%|d}n|j|j|j}}}tj|jƒ}tj|j|jfƒ}xLt|ƒD]>}tj|j dƒƒ}|dd…|gf} d} d} x¦| |krg| |krg|j j | ƒ| j j | ƒ} | tj | j j | ƒƒ} | } |j | ƒ| j j | ƒ} t | | ƒt | ƒ} | d7} qÂW| djƒ||<| |dd…|gf<|dkr||| j | j ƒ8}q|q|W||_||_dS(sg NIPALS implementation to compute small number of eigenvalues and eigenvectors igiNgð?i(R7R*RRRtzerosR#RtargmaxRkR‡RŒRR RR@RA(RJRRRRSRKtvalstvecstit max_var_indtfactort_itertdifftvect factor_last((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyR…´s0  "  cC sëtjtj|jƒƒ}tj|ƒr1|jStj|jƒƒ}|_|j}tj |dƒ}tj |dƒ}tj ||kƒs¤tj ||kƒr³t dƒ‚ntj|ƒ}tj |dƒ}tj |jdfƒ|}||} | ||R‡R?RR(RJRRtindicestnum_goodR+((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyRp s0       cC sH|j}|jtj|ƒ}tj|ddƒ|_tj|jƒ|_tj|jdƒ|_ tj|jd|j fƒ|_ xt |jdƒD]z}|j d|dtdtƒ}|djddƒ}|jƒ}|j||j |<|j||j |dd…ftSeriesR@tnametreplaceRAR!RCRRD(RJRt num_zerostcomp_strR‘R1tdftvec_str((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyRIŠs4 +       2 $c C s ddljj}|j|ƒ\}}|dkr<|jn|}tj|jƒ}||j }|rytj |ƒ}n|r|j dƒn|j tj |ƒ|| dƒ|j dtƒtj|jƒƒ}|d|d} |dtj| | gƒ7}|j|ƒtj|jƒƒ} d} |r˜tj| d| dƒ} tjtjtj| dƒ| | tj| dƒ| | gƒƒ} n0| d| d} | | tj| | gƒ7} |j| ƒ|jd ƒ|jd ƒ|jd ƒ|jƒ|S( sØ Plot of the ordered eigenvalues Parameters ---------- ncomp : int, optional Number of components ot include in the plot. If None, will included the same as the number of components computed log_scale : boot, optional Flag indicating whether ot use a log scale for the y-axis cumulative : bool, optional Flag indicating whether to plot the eigenvalues or cumulative eigenvalues ax : Matplotlib axes instance, optional An axes on which to draw the graph. If omitted, new a figure is created Returns ------- fig : figure Handle to the figure iÿÿÿÿNRªtbottightiig{®Gáz”?s Scree PlotRxsComponent Number(tstatsmodels.graphics.utilstgraphicstutilst create_mpl_axRR*RRR@tcumsumt set_yscaletplotR/t autoscaletTrueR%tget_xlimtset_xlimtget_ylimRªtexptset_ylimt set_titlet set_ylabelt set_xlabelt tight_layout( RJRKt log_scalet cumulativetaxtgutilstfigRtxlimtsptylimtscale((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyt plot_scree±s:   $'     cC s´ddljj}|j|ƒ\}}|dkr9dn|}t||jƒ}d|j|j}|d}|| }|j |j ƒ|j dƒ|j dƒ|j dƒ|S( s Box plots of the individual series R-square against the number of PCs Parameters ---------- ncomp : int, optional Number of components ot include in the plot. If None, will plot the minimum of 10 or the number of computed components ax : Matplotlib axes instance, optional An axes on which to draw the graph. If omitted, new a figure is created Returns ------- fig : figure Handle to the figure iÿÿÿÿNi gð?isIndividual Input $R^2$s$R^2$s'Number of Included Principal Components(RÆRÇRÈRÉRR)R*R:R;tboxplotR‡RÔRÕRÖ(RJRKRÚRÛRÜtr2s((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyt plot_rsquareís     N(t__name__t __module__t__doc__RRÎRšRWR4RGRFRwRsRERoR„RƒR…R_RpRHRqRIRáRä(((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyR s.£    a 0        9 $ )0 ';R c C sdt|d|d|d|d|d|d|d|ƒ}|j|j|j|j|j|j|jfS(s/ Principal Component Analysis Parameters ---------- data : array Variables in columns, observations in rows. ncomp : int, optional Number of components to return. If None, returns the as many as the smaller to the number of rows or columns of data. standardize: bool, optional Flag indicating to use standardized data with mean 0 and unit variance. standardized being True implies demean. demean : bool, optional Flag indicating whether to demean data before computing principal components. demean is ignored if standardize is True. normalize : bool , optional Indicates whether th normalize the factors to have unit inner product. If False, the loadings will have unit inner product. weights : array, optional Series weights to use after transforming data according to standardize or demean when computing the principal components. gls : bool, optional Flag indicating to implement a two-step GLS estimator where in the first step principal components are used to estimate residuals, and then the inverse residual variance is used as a set of weights to estimate the final principal components method : str, optional Determines the linear algebra routine uses. 'eig', the default, uses an eigenvalue decomposition. 'svd' uses a singular value decomposition. Returns ------- factors : array or DataFrame nobs by ncomp array of of principal components (also known as scores) loadings : array or DataFrame ncomp by nvar array of principal component loadings for constructing the factors projection : array or DataFrame nobs by var array containing the projection of the data onto the ncomp estimated factors rsquare : array or Series ncomp array where the element in the ith position is the R-square of including the fist i principal components. The values are calculated on the transformed data, not the original data. ic : array or DataFrame ncomp by 3 array containing the Bai and Ng (2003) Information criteria. Each column is a different criteria, and each row represents the number of included factors. eigenvals : array or Series nvar array of eigenvalues eigenvecs : array or DataFrame nvar by nvar array of eigenvectors Notes ----- This is a simple function wrapper around the PCA class. See PCA for more information and additional methods. RKRLRMRNROR(RP(R R=R>RBRCRDR@RA( RRKRLRMRNROR(RPtpc((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pytpcas>(Rçt __future__RRtnumpyRtpandasRtstatsmodels.compat.pythonRtstatsmodels.tools.sm_exceptionsRRR tobjectR RRÎRšRé(((s;lib/python2.7/site-packages/statsmodels/multivariate/pca.pyts   ÿÿÿ