ó ‡ˆ\c @sXdZddlZddlZddlmZddlmZmZddl m Z ddl m Z ddl mZdd l mZdd lmZmZmZdd lmZdd lmZd dgZd„Zd„Zd„Zd„Zdd„Zd„Zd„Zdde dddddde!e e!d„ Z"deefd„ƒYZ#dS(s Python implementation of the fast ICA algorithms. Reference: Tables 8.3 and 8.4 page 196 in the book: Independent Component Analysis, by Hyvarinen et al. iÿÿÿÿN(tlinalgi(t BaseEstimatortTransformerMixin(tConvergenceWarning(tsix(tmoves(t string_types(t check_arraytas_float_arraytcheck_random_state(tcheck_is_fitted(t FLOAT_DTYPEStfasticatFastICAcCs1|tjtj||| jƒ|| ƒ8}|S(s Orthonormalize w wrt the first j rows of W Parameters ---------- w : ndarray of shape(n) Array to be orthogonalized W : ndarray of shape(p, n) Null space definition j : int < p The no of (from the first) rows of Null space W wrt which w is orthogonalized. Notes ----- Assumes that W is orthogonal w changed in place (tnptdottT(twtWtj((s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pyt_gs_decorrelations-cCsTtjtj||jƒƒ\}}tjtj|dtj|ƒ|jƒ|ƒS(sA Symmetric decorrelation i.e. W <- (W * W.T) ^{-1/2} * W gð?(RteighRRRtsqrt(Rtstu((s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pyt_sym_decorrelation6s$cCsƒ|jd}tj||fd|jƒ}g}x?t|ƒD]1} || dd…fjƒ} | tj| djƒƒ:} xÈtj |ƒD]·} |tj | j |ƒ|ƒ\} } || j ddƒ| j ƒ| }t ||| ƒ|tj|djƒƒ:}tjtj|| jƒƒdƒ}|} ||krPqqW|j| dƒ| || dd…fW|t|ƒfS(scDeflationary FastICA using fun approx to neg-entropy function Used internally by FastICA. itdtypeNitaxisi(tshapeRtzerosRtrangetcopyRtsumRtxrangeRRtmeanRtabstappendtmax(tXttoltgtfun_argstmax_itertw_initt n_componentsRtn_iterRRtitgwtxtg_wtxtw1tlim((s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pyt_ica_def@s$ $$& c Cst|ƒ}~t|jdƒ}xÒtj|ƒD]±}|tj||ƒ|ƒ\} } ttj| |jƒ|| dd…tjf|ƒ} ~ ~ t t t tj tj| |jƒƒƒdƒƒ} | }| |kr2Pq2q2Wt j dtƒ||dfS(sCParallel FastICA. Used internally by FastICA --main loop iNs\FastICA did not converge. Consider increasing tolerance or the maximum number of iterations.(RtfloatRRR!RRRtnewaxisR%R#tdiagtwarningstwarnR( R&R'R(R)R*R+Rtp_tiiR/R0tW1R2((s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pyt_ica_parcs !!4  cCs‡|jddƒ}||9}tj||ƒ}tj|jdƒ}x6t|ƒD](\}}|d|djƒ||`. Parameters ---------- X : array-like, shape (n_samples, n_features) Training vector, where n_samples is the number of samples and n_features is the number of features. n_components : int, optional Number of components to extract. If None no dimension reduction is performed. algorithm : {'parallel', 'deflation'}, optional Apply a parallel or deflational FASTICA algorithm. whiten : boolean, optional If True perform an initial whitening of the data. If False, the data is assumed to have already been preprocessed: it should be centered, normed and white. Otherwise you will get incorrect results. In this case the parameter n_components will be ignored. fun : string or function, optional. Default: 'logcosh' The functional form of the G function used in the approximation to neg-entropy. Could be either 'logcosh', 'exp', or 'cube'. You can also provide your own function. It should return a tuple containing the value of the function, and of its derivative, in the point. The derivative should be averaged along its last dimension. Example: def my_g(x): return x ** 3, np.mean(3 * x ** 2, axis=-1) fun_args : dictionary, optional Arguments to send to the functional form. If empty or None and if fun='logcosh', fun_args will take value {'alpha' : 1.0} max_iter : int, optional Maximum number of iterations to perform. tol : float, optional A positive scalar giving the tolerance at which the un-mixing matrix is considered to have converged. w_init : (n_components, n_components) array, optional Initial un-mixing array of dimension (n.comp,n.comp). If None (default) then an array of normal r.v.'s is used. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. return_X_mean : bool, optional If True, X_mean is returned too. compute_sources : bool, optional If False, sources are not computed, but only the rotation matrix. This can save memory when working with big data. Defaults to True. return_n_iter : bool, optional Whether or not to return the number of iterations. Returns ------- K : array, shape (n_components, n_features) | None. If whiten is 'True', K is the pre-whitening matrix that projects data onto the first n_components principal components. If whiten is 'False', K is 'None'. W : array, shape (n_components, n_components) Estimated un-mixing matrix. The mixing matrix can be obtained by:: w = np.dot(W, K.T) A = w.T * (w * w.T).I S : array, shape (n_samples, n_components) | None Estimated source matrix X_mean : array, shape (n_features, ) The mean over features. Returned only if return_X_mean is True. n_iter : int If the algorithm is "deflation", n_iter is the maximum number of iterations run across all components. Else they are just the number of iterations taken to converge. This is returned only when return_n_iter is set to `True`. Notes ----- The data matrix X is considered to be a linear combination of non-Gaussian (independent) components i.e. X = AS where columns of S contain the independent components and A is a linear mixing matrix. In short ICA attempts to `un-mix' the data by estimating an un-mixing matrix W where ``S = W K X.`` This implementation was originally made for data of shape [n_features, n_samples]. Now the input is transposed before the algorithm is applied. This makes it slightly faster for Fortran-ordered input. Implemented using FastICA: `A. Hyvarinen and E. Oja, Independent Component Analysis: Algorithms and Applications, Neural Networks, 13(4-5), 2000, pp. 411-430` RRtensure_min_samplesiR=gð?isalpha must be in [1,2]RKRGtcubecs ˆ||S(N((RBR)(tfun(s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pyR(ssJUnknown function %r; should be one of 'logcosh', 'exp', 'cube' or callables(Ignoring n_components with whiten=False.s/n_components is too large: it will be set to %sRiÿÿÿÿNt full_matricestsizes/w_init has invalid shape -- should be %(shape)sRR'R(R)R*R+RJt deflations<Invalid algorithm: must be either `parallel` or `deflation`.(!R tNoneRR RR>t ValueErrorRFRHRItcallablet isinstanceRRt TypeErrorRR7R8tminR"RR5RtsvdtFalseRRRtasarraytnormalRR<R3(R&R,t algorithmtwhitenRNR)R*R'R+t random_statet return_X_meantcompute_sourcest return_n_iterR=R(texctntptX_meanRtdt_tKtX1tkwargsRR-tS((RNs=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pyR —s˜v                 $c BsqeZdZd dedd ddd d d„ Zed„Zd d„Zd d„Z d ed „Z ed „Z RS( sŠ FastICA: a fast algorithm for Independent Component Analysis. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, optional Number of components to use. If none is passed, all are used. algorithm : {'parallel', 'deflation'} Apply parallel or deflational algorithm for FastICA. whiten : boolean, optional If whiten is false, the data is already considered to be whitened, and no whitening is performed. fun : string or function, optional. Default: 'logcosh' The functional form of the G function used in the approximation to neg-entropy. Could be either 'logcosh', 'exp', or 'cube'. You can also provide your own function. It should return a tuple containing the value of the function, and of its derivative, in the point. Example: def my_g(x): return x ** 3, (3 * x ** 2).mean(axis=-1) fun_args : dictionary, optional Arguments to send to the functional form. If empty and if fun='logcosh', fun_args will take value {'alpha' : 1.0}. max_iter : int, optional Maximum number of iterations during fit. tol : float, optional Tolerance on update at each iteration. w_init : None of an (n_components, n_components) ndarray The mixing matrix to be used to initialize the algorithm. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Attributes ---------- components_ : 2D array, shape (n_components, n_features) The unmixing matrix. mixing_ : array, shape (n_features, n_components) The mixing matrix. n_iter_ : int If the algorithm is "deflation", n_iter is the maximum number of iterations run across all components. Else they are just the number of iterations taken to converge. Examples -------- >>> from sklearn.datasets import load_digits >>> from sklearn.decomposition import FastICA >>> X, _ = load_digits(return_X_y=True) >>> transformer = FastICA(n_components=7, ... random_state=0) >>> X_transformed = transformer.fit_transform(X) >>> X_transformed.shape (1797, 7) Notes ----- Implementation based on `A. Hyvarinen and E. Oja, Independent Component Analysis: Algorithms and Applications, Neural Networks, 13(4-5), 2000, pp. 411-430` RJRKiÈg-Cëâ6?c CsŒtt|ƒjƒ|dkr7tdj|ƒƒ‚n||_||_||_||_||_ ||_ ||_ ||_ | |_ dS(Nis4max_iter should be greater than 1, got (max_iter={})(tsuperR t__init__RStformatR,R\R]RNR)R*R'R+R^( tselfR,R\R]RNR)R*R'R+R^((s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pyRmÑs          cCs|jdkrin|j}td|d|jd|jd|jd|jd|d|jd|jd |j d |j d t d |d t ƒ \}}}}|_ |jrÒt j||ƒ|_||_||_n ||_tj|jƒ|_|r||_n|S(s Fit the model Parameters ---------- X : array-like, shape (n_samples, n_features) Training data, where n_samples is the number of samples and n_features is the number of features. compute_sources : bool If False, sources are not computes but only the rotation matrix. This can save memory when working with big data. Defaults to False. Returns ------- X_new : array-like, shape (n_samples, n_components) R&R,R\R]RNR)R*R'R+R^R_R`RaN(R)RRR R,R\R]RNR*R'R+R^tTruetn_iter_RRt components_tmean_t whitening_Rtpinvtmixing_t_FastICA__sources(RoR&R`R)t whiteningtunmixingtsourcesRe((s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pyt_fitâs !     cCs|j|dtƒS(sFit the model and recover the sources from X. Parameters ---------- X : array-like, shape (n_samples, n_features) Training data, where n_samples is the number of samples and n_features is the number of features. y : Ignored Returns ------- X_new : array-like, shape (n_samples, n_components) R`(R{Rp(RoR&ty((s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pyt fit_transform scCs|j|dtƒ|S(s6Fit the model to X. Parameters ---------- X : array-like, shape (n_samples, n_features) Training data, where n_samples is the number of samples and n_features is the number of features. y : Ignored Returns ------- self R`(R{RY(RoR&R|((s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pytfitst deprecatedcCsƒt|tƒ s|dkr/tjdtƒnt|dƒt|d|dtƒ}|jrm||j 8}nt j ||j j ƒS(sQRecover the sources from X (apply the unmixing matrix). Parameters ---------- X : array-like, shape (n_samples, n_features) Data to transform, where n_samples is the number of samples and n_features is the number of features. y : (ignored) .. deprecated:: 0.19 This parameter will be removed in 0.21. copy : bool (optional) If False, data passed to fit are overwritten. Defaults to True. Returns ------- X_new : array-like, shape (n_samples, n_components) RsSThe parameter y on transform() is deprecated since 0.19 and will be removed in 0.21RvRR(RURR7R8tDeprecationWarningR RR R]RsRRRrR(RoR&R|R((s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pyt transform,s    cCsct|dƒt|d|o"|jdtƒ}tj||jjƒ}|jr_||j7}n|S(séTransform the sources back to the mixed data (apply mixing matrix). Parameters ---------- X : array-like, shape (n_samples, n_components) Sources, where n_samples is the number of samples and n_components is the number of components. copy : bool (optional) If False, data passed to fit are overwritten. Defaults to True. Returns ------- X_new : array-like, shape (n_samples, n_features) RvRR( R RR]R RRRvRRs(RoR&R((s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pytinverse_transformKs  ! N( t__name__t __module__t__doc__RRRpRmRYR{R}R~RR‚(((s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pyR sO   '  ($R…R7tnumpyRtscipyRtbaseRRt exceptionsRt externalsRt externals.sixRRtutilsRRR tutils.validationR R t__all__RRR3R<RRRFRHRIRpRYR R (((s=lib/python2.7/site-packages/sklearn/decomposition/fastica_.pyts0     #      ç