B \M @sdZddlZddlZddlmZddlmZmZddl m Z ddl m Z ddl mZdd l mZdd lmZmZmZdd lmZdd lmZd dgZddZddZddZddZd%ddZddZddZd&d#d ZGd$ddeeZ dS)'z Python implementation of the fast ICA algorithms. Reference: Tables 8.3 and 8.4 page 196 in the book: Independent Component Analysis, by Hyvarinen et al. N)linalg) BaseEstimatorTransformerMixin)ConvergenceWarning)six)moves) string_types) check_arrayas_float_arraycheck_random_state)check_is_fitted) FLOAT_DTYPESfasticaFastICAc Cs.|tt||d|j|d|8}|S)a Orthonormalize w wrt the first j rows of W Parameters ---------- w : ndarray of shape(n) Array to be orthogonalized W : ndarray of shape(p, n) Null space definition j : int < p The no of (from the first) rows of Null space W wrt which w is orthogonalized. Notes ----- Assumes that W is orthogonal w changed in place N)npdotT)wWjr=lib/python3.7/site-packages/sklearn/decomposition/fastica_.py_gs_decorrelations*rc Cs<tt||j\}}tt|dt||j|S)zA Symmetric decorrelation i.e. W <- (W * W.T) ^{-1/2} * W g?)rZeighrrrsqrt)rsurrr_sym_decorrelation6srcCs|jd}tj||f|jd}g}xt|D]} || ddf} | t| d} xt |D]} |t | j ||\} } || j dd| | }t ||| |t|d}tt|| d}|} ||krfPqfW|| d| || ddf<q,W|t|fS)zcDeflationary FastICA using fun approx to neg-entropy function Used internally by FastICA. r)dtypeNr)axis)shaperZzerosrrangecopyrsumrxrangerrmeanrabsappendmax)Xtolgfun_argsmax_iterw_init n_componentsrn_iterrrigwtxg_wtxZw1limrrr_ica_def@s$  r6c Cst|}~t|jd}xt|D]|}|t|||\} } tt| |j|| ddtjf|} ~ ~ t t t t t| |jd} | }| |kr$Pq$Wt dt||dfS)zCParallel FastICA. Used internally by FastICA --main loop rNz\FastICA did not converge. Consider increasing tolerance or the maximum number of iterations.)rfloatr!rr%rrrnewaxisr)r'Zdiagwarningswarnr) r*r+r,r-r.r/rZp_Ziir3r4ZW1r5rrr_ica_parcs$r;cCsf|dd}||9}t||}t|jd}x,t|D] \}}|d|d||<q:W||fS)Nalphag?rrr)getrZtanhemptyr! enumerater&)xr-r<gxg_xr2Zgx_irrr_logcoshs  rCcCs<t|d d}||}d|d|}||jddfS)Nrr)r )rexpr&)r@r-rErArBrrr_expsrFcCs|dd|djddfS)NrrD)r )r&)r@r-rrr_cubesrHparallelTlogcosh-C6?Fc st| } |dkrin|}t||tddj}|dd} d| krJdksTntddkrbt}nRd krpt}nDd kr~t}n6t rfd d }n t t j rtnt }|d |j\}}|s|dk rd}td|dkrt||}|t||krt||}td||r|jdd}||ddtjf8}tj|dd\}}}~||jd|}~~t||}|t|9}n t|dd}|dkrtj| j||fd|jd}n.t|}|j||fkrtdd||fi|||||d}|dkrt|f|\}}n$|dkr,t|f|\}}ntd~|r| rZtt|||j}nd}| r| rx|||||fS||||fSn| r||||fS|||fSn^| rt||j}nd}| r| rd||d|fSd||dfSn| rd|||fSd||fSdS)aPerform Fast Independent Component Analysis. Read more in the :ref:`User Guide `. Parameters ---------- X : array-like, shape (n_samples, n_features) Training vector, where n_samples is the number of samples and n_features is the number of features. n_components : int, optional Number of components to extract. If None no dimension reduction is performed. algorithm : {'parallel', 'deflation'}, optional Apply a parallel or deflational FASTICA algorithm. whiten : boolean, optional If True perform an initial whitening of the data. If False, the data is assumed to have already been preprocessed: it should be centered, normed and white. Otherwise you will get incorrect results. In this case the parameter n_components will be ignored. fun : string or function, optional. Default: 'logcosh' The functional form of the G function used in the approximation to neg-entropy. Could be either 'logcosh', 'exp', or 'cube'. You can also provide your own function. It should return a tuple containing the value of the function, and of its derivative, in the point. The derivative should be averaged along its last dimension. Example: def my_g(x): return x ** 3, np.mean(3 * x ** 2, axis=-1) fun_args : dictionary, optional Arguments to send to the functional form. If empty or None and if fun='logcosh', fun_args will take value {'alpha' : 1.0} max_iter : int, optional Maximum number of iterations to perform. tol : float, optional A positive scalar giving the tolerance at which the un-mixing matrix is considered to have converged. w_init : (n_components, n_components) array, optional Initial un-mixing array of dimension (n.comp,n.comp). If None (default) then an array of normal r.v.'s is used. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. return_X_mean : bool, optional If True, X_mean is returned too. compute_sources : bool, optional If False, sources are not computed, but only the rotation matrix. This can save memory when working with big data. Defaults to True. return_n_iter : bool, optional Whether or not to return the number of iterations. Returns ------- K : array, shape (n_components, n_features) | None. If whiten is 'True', K is the pre-whitening matrix that projects data onto the first n_components principal components. If whiten is 'False', K is 'None'. W : array, shape (n_components, n_components) Estimated un-mixing matrix. The mixing matrix can be obtained by:: w = np.dot(W, K.T) A = w.T * (w * w.T).I S : array, shape (n_samples, n_components) | None Estimated source matrix X_mean : array, shape (n_features, ) The mean over features. Returned only if return_X_mean is True. n_iter : int If the algorithm is "deflation", n_iter is the maximum number of iterations run across all components. Else they are just the number of iterations taken to converge. This is returned only when return_n_iter is set to `True`. Notes ----- The data matrix X is considered to be a linear combination of non-Gaussian (independent) components i.e. X = AS where columns of S contain the independent components and A is a linear mixing matrix. In short ICA attempts to `un-mix' the data by estimating an un-mixing matrix W where ``S = W K X.`` This implementation was originally made for data of shape [n_features, n_samples]. Now the input is transposed before the algorithm is applied. This makes it slightly faster for Fortran-ordered input. Implemented using FastICA: `A. Hyvarinen and E. Oja, Independent Component Analysis: Algorithms and Applications, Neural Networks, 13(4-5), 2000, pp. 411-430` Nr)r#rZensure_min_samplesr<g?rzalpha must be in [1,2]rJrEZcubecs |f|S)Nr)r@r-)funrrr,szfastica..gzJUnknown function %r; should be one of 'logcosh', 'exp', 'cube' or callablez(Ignoring n_components with whiten=False.z/n_components is too large: it will be set to %srD)r F)Z full_matrices)r#)size)rz/w_init has invalid shape -- should be %(shape)sr!)r+r,r-r.r/rIZ deflationz`. Parameters ---------- n_components : int, optional Number of components to use. If none is passed, all are used. algorithm : {'parallel', 'deflation'} Apply parallel or deflational algorithm for FastICA. whiten : boolean, optional If whiten is false, the data is already considered to be whitened, and no whitening is performed. fun : string or function, optional. Default: 'logcosh' The functional form of the G function used in the approximation to neg-entropy. Could be either 'logcosh', 'exp', or 'cube'. You can also provide your own function. It should return a tuple containing the value of the function, and of its derivative, in the point. Example: def my_g(x): return x ** 3, (3 * x ** 2).mean(axis=-1) fun_args : dictionary, optional Arguments to send to the functional form. If empty and if fun='logcosh', fun_args will take value {'alpha' : 1.0}. max_iter : int, optional Maximum number of iterations during fit. tol : float, optional Tolerance on update at each iteration. w_init : None of an (n_components, n_components) ndarray The mixing matrix to be used to initialize the algorithm. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Attributes ---------- components_ : 2D array, shape (n_components, n_features) The unmixing matrix. mixing_ : array, shape (n_features, n_components) The mixing matrix. n_iter_ : int If the algorithm is "deflation", n_iter is the maximum number of iterations run across all components. Else they are just the number of iterations taken to converge. Examples -------- >>> from sklearn.datasets import load_digits >>> from sklearn.decomposition import FastICA >>> X, _ = load_digits(return_X_y=True) >>> transformer = FastICA(n_components=7, ... random_state=0) >>> X_transformed = transformer.fit_transform(X) >>> X_transformed.shape (1797, 7) Notes ----- Implementation based on `A. Hyvarinen and E. Oja, Independent Component Analysis: Algorithms and Applications, Neural Networks, 13(4-5), 2000, pp. 411-430` NrITrJrK-C6?c s^tt||dkr$td|||_||_||_||_||_ ||_ ||_ ||_ | |_ dS)Nrz4max_iter should be greater than 1, got (max_iter={}))superr__init__rOformatr0rTrUrMr-r.r+r/rV) selfr0rTrUrMr-r.r+r/rV) __class__rrreszFastICA.__init__FcCs|jdkrin|j}t||j|j|j|j||j|j|j|j d|dd \}}}}|_ |jrtt |||_ ||_||_n||_ t|j |_|r||_|S)a Fit the model Parameters ---------- X : array-like, shape (n_samples, n_features) Training data, where n_samples is the number of samples and n_features is the number of features. compute_sources : bool If False, sources are not computes but only the rotation matrix. This can save memory when working with big data. Defaults to False. Returns ------- X_new : array-like, shape (n_samples, n_components) NT) r*r0rTrUrMr-r.r+r/rVrWrXrY)r-rr0rTrUrMr.r+r/rVZn_iter_rr components_mean_Z whitening_rZpinvmixing_Z_FastICA__sources)rgr*rXr-Z whiteningZunmixingZsourcesr]rrr_fits    z FastICA._fitcCs|j|ddS)aFit the model and recover the sources from X. Parameters ---------- X : array-like, shape (n_samples, n_features) Training data, where n_samples is the number of samples and n_features is the number of features. y : Ignored Returns ------- X_new : array-like, shape (n_samples, n_components) T)rX)rl)rgr*yrrr fit_transform szFastICA.fit_transformcCs|j|dd|S)a6Fit the model to X. Parameters ---------- X : array-like, shape (n_samples, n_features) Training data, where n_samples is the number of samples and n_features is the number of features. y : Ignored Returns ------- self F)rX)rl)rgr*rmrrrfitsz FastICA.fit deprecatedcCsVt|tr|dkrtdtt|dt||td}|jrF||j 8}t ||j j S)aQRecover the sources from X (apply the unmixing matrix). Parameters ---------- X : array-like, shape (n_samples, n_features) Data to transform, where n_samples is the number of samples and n_features is the number of features. y : (ignored) .. deprecated:: 0.19 This parameter will be removed in 0.21. copy : bool (optional) If False, data passed to fit are overwritten. Defaults to True. Returns ------- X_new : array-like, shape (n_samples, n_components) rpzSThe parameter y on transform() is deprecated since 0.19 and will be removed in 0.21rk)r#r)rQr r9r:DeprecationWarningr r rrUrjrrrir)rgr*rmr#rrr transform,s  zFastICA.transformcCsBt|dt||o|jtd}t||jj}|jr>||j7}|S)aTransform the sources back to the mixed data (apply mixing matrix). Parameters ---------- X : array-like, shape (n_samples, n_components) Sources, where n_samples is the number of samples and n_components is the number of components. copy : bool (optional) If False, data passed to fit are overwritten. Defaults to True. Returns ------- X_new : array-like, shape (n_samples, n_features) rk)r#r) r r rUrrrrkrrj)rgr*r#rrrinverse_transformKs   zFastICA.inverse_transform) NrITrJNrKrcNN)F)N)N)rpT)T) __name__ __module__ __qualname____doc__rerlrnrorrrs __classcell__rr)rhrrsO '   )N) NrITrJNrKrLNNFTF)!rwr9ZnumpyrZscipyrbaserr exceptionsrZ externalsrZ externals.sixrr Zutilsr r r Zutils.validationr r__all__rrr6r;rCrFrHrrrrrrs2        #  h