\c@ sBdZddlmZmZddlmZddlZddlZddlZddl Z ddl j Z ddlmZmZddlmZmZddlmZmZmZdd lmZdd lmZmZdd lmZd d lmZe j e j!j"Z#ej$e j%fZ&dZ'dZ(dZ)e*dZ+dZ,dZ-dZ.dZ/ddddZ1dZ2dddddde3de*dd Z4ddde3dZ5dZ6ddddddde3dd Z7dddd!e3d"dddd#d#ddde*d$Z8d%eefd&YZ9dS('s# Non-negative matrix factorization i(tdivisiontprint_function(tsqrtNi(t BaseEstimatortTransformerMixin(tcheck_random_statet check_array(trandomized_svdtsafe_sparse_dott squared_norm(tsafe_min(tcheck_is_fittedtcheck_non_negative(tConvergenceWarningi(t_update_cdnmf_fastcC stt|S(sDot product-based Euclidean norm implementation See: http://fseoane.net/blog/2011/computing-the-vector-norm/ Parameters ---------- x : array-like Vector for which to compute the norm (RR (tx((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pytnorm!s cC stj|j|jS(sTrace of np.dot(X, Y.T). Parameters ---------- X : array-like First matrix Y : array-like Second matrix (tnptdottravel(tXtY((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pyt trace_dot.s cC st|}tj||krFtd||tj|fnt||tj|dkr{td|ndS(Ns=Array with wrong shape passed to %s. Expected %s, but got %s is$Array passed to %s is full of zeros.(RRtshapet ValueErrorR tmax(tARtwhom((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pyt _check_init;s  cC sVt|}tj|s-tj|}ntj|}tj|}|dkrtj|rtj|j|j}ttjtj|j|||}t||j|}||d|d}n t |tj||d}|rtj |dS|Sntj|rEt |||j} |j} n*tj||} | j } |j } | t k} | | } | | } t | | dk<|dkrtjtj|ddtj|dd} | | }tj| tj|}|| | j7}n |dkre| | }tj|tj|jtjtj|}ntj|rd}xdt|jdD]9}|tjtj||dd|f|7}qWntj| |}tj| | |d}| |j||}|||d7}|||d}|rNtj d|S|SdS(s{Compute the beta-divergence of X and dot(W, H). Parameters ---------- X : float or array-like, shape (n_samples, n_features) W : float or dense array-like, shape (n_samples, n_components) H : float or dense array-like, shape (n_components, n_features) beta : float, string in {'frobenius', 'kullback-leibler', 'itakura-saito'} Parameter of the beta-divergence. If beta == 2, this is half the Frobenius *squared* norm. If beta == 1, this is the generalized Kullback-Leibler divergence. If beta == 0, this is the Itakura-Saito divergence. Else, this is the general beta-divergence. square_root : boolean, default False If True, return np.sqrt(2 * res) For beta == 2, it corresponds to the Frobenius norm. Returns ------- res : float Beta divergence of X and np.dot(X, H) ig@iitaxisN(t_beta_loss_to_floattsptissparseRt atleast_2dRtdataRtTR Rt_special_sparse_dotRtEPSILONtsumtlogtproductRtrange(RtWtHtbetat square_roottnorm_Xtnorm_WHt cross_prodtrestWH_datatX_datatWHtindicestsum_WHtdivt sum_WH_betatitsum_X_WH((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pyt_beta_divergenceEsZ  *        0   8:cC stj|r|j\}}tj||ddf|j|ddfjdd}tj|||ffd|j}|j Stj ||SdS(s0Computes np.dot(W, H), only where X is non zero.NRiR( RR tnonzeroRtmultiplyR#R&t coo_matrixRttocsrR(R*R+Rtiitjjtdot_valsR4((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pyR$s A$ c C sd}d}|dkr't|}n|dkrBt|}n||}||}|d|}|d|}||||fS(s9Compute L1 and L2 regularization coefficients for W and Hgtbotht componentsttransformationg?(RCRD(RCRE(tfloat( talphatl1_ratiotregularizationtalpha_Htalpha_Wtl1_reg_Wtl1_reg_Htl2_reg_Wtl2_reg_H((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pyt_compute_regularizations    cC sd }||kr+td||fnd}||krVtd||fn|dkr|dkrtd ||fn|dkr|d krtjd tnt|}|S(Ntcdtmus5Invalid solver parameter: got %r instead of one of %rRCRDREs=Invalid regularization parameter: got %r instead of one of %rit frobeniussEInvalid beta_loss parameter: solver %r does not handle beta_loss = %rtnndsvdsThe multiplicative update ('mu') solver cannot update zeros present in the initialization, and so leads to poorer results when used jointly with init='nndsvd'. You may try init='nndsvda' or init='nndsvdar' instead.(RQRR(RCRDREN(iRS(RtNonetwarningstwarnt UserWarningR(tsolverRIt beta_losstinittallowed_solvertallowed_regularization((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pyt_check_string_params&     cC sxidd6dd6dd6}t|trC||krC||}nt|tjsttd||jfn|S(s!Convert string beta_loss to floatiRSiskullback-leibleris itakura-saitosEInvalid beta_loss parameter: got %r instead of one of %r, or a float.(t isinstancetstrtnumberstNumberRtkeys(RZtallowed_beta_loss((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pyRs   gư>cC s't|d|j\}}|dkrF||kr=d}qFd}n|dkrtj|j|}t|}||j||} ||j||} tj| | tj| | | | fSt ||d|\} } } tj | jtj | j} } tj| dtj| dddf| dddf>> import numpy as np >>> X = np.array([[1,1], [2, 1], [3, 1.2], [4, 1], [5, 0.8], [6, 1]]) >>> from sklearn.decomposition import non_negative_factorization >>> W, H, n_iter = non_negative_factorization(X, n_components=2, ... init='random', random_state=0) References ---------- Cichocki, Andrzej, and P. H. A. N. Anh-Huy. "Fast local algorithms for large scale nonnegative matrix and tensor factorizations." IEICE transactions on fundamentals of electronics, communications and computer sciences 92.3: 708-721, 2009. Fevotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Computation, 23(9). RRtcscRs NMF (input X)is|When beta_loss <= 0 and X contains zeros, the solver may diverge. Please add small values to X, or use a positive beta_loss.sFNumber of components must be a positive integer; got (n_components=%r)sJMaximum number of iterations must be a positive integer; got (max_iter=%r)s>Tolerance for stopping criteria must be positive; got (tol=%r)tcustoms NMF (input H)s NMF (input W)RRR[RfRQRRRsInvalid solver parameter '%s'.sKMaximum number of iteration %d reached. Increase it to improve convergence.(RRN(RRFR R^R RRRUR_t INTEGER_TYPESRaRbRRRRitfullRlRRPRRRVRWR (RR*R+RpR[RRYRZRRRGRHRIRfRRRrRsRtRLRMRNROR((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pytnon_negative_factorizationAs`             tNMFc B sheZdZd d ddddd ddded Zd d d dZd d Zd Zd Z RS( sNon-Negative Matrix Factorization (NMF) Find two non-negative matrices (W, H) whose product approximates the non- negative matrix X. This factorization can be used for example for dimensionality reduction, source separation or topic extraction. The objective function is:: 0.5 * ||X - WH||_Fro^2 + alpha * l1_ratio * ||vec(W)||_1 + alpha * l1_ratio * ||vec(H)||_1 + 0.5 * alpha * (1 - l1_ratio) * ||W||_Fro^2 + 0.5 * alpha * (1 - l1_ratio) * ||H||_Fro^2 Where:: ||A||_Fro^2 = \sum_{i,j} A_{ij}^2 (Frobenius norm) ||vec(A)||_1 = \sum_{i,j} abs(A_{ij}) (Elementwise L1 norm) For multiplicative-update ('mu') solver, the Frobenius norm (0.5 * ||X - WH||_Fro^2) can be changed into another beta-divergence loss, by changing the beta_loss parameter. The objective function is minimized with an alternating minimization of W and H. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int or None Number of components, if n_components is not set all features are kept. init : 'random' | 'nndsvd' | 'nndsvda' | 'nndsvdar' | 'custom' Method used to initialize the procedure. Default: 'nndsvd' if n_components < n_features, otherwise random. Valid options: - 'random': non-negative random matrices, scaled with: sqrt(X.mean() / n_components) - 'nndsvd': Nonnegative Double Singular Value Decomposition (NNDSVD) initialization (better for sparseness) - 'nndsvda': NNDSVD with zeros filled with the average of X (better when sparsity is not desired) - 'nndsvdar': NNDSVD with zeros filled with small random values (generally faster, less accurate alternative to NNDSVDa for when sparsity is not desired) - 'custom': use custom matrices W and H solver : 'cd' | 'mu' Numerical solver to use: 'cd' is a Coordinate Descent solver. 'mu' is a Multiplicative Update solver. .. versionadded:: 0.17 Coordinate Descent solver. .. versionadded:: 0.19 Multiplicative Update solver. beta_loss : float or string, default 'frobenius' String must be in {'frobenius', 'kullback-leibler', 'itakura-saito'}. Beta divergence to be minimized, measuring the distance between X and the dot product WH. Note that values different from 'frobenius' (or 2) and 'kullback-leibler' (or 1) lead to significantly slower fits. Note that for beta_loss <= 0 (or 'itakura-saito'), the input matrix X cannot contain zeros. Used only in 'mu' solver. .. versionadded:: 0.19 tol : float, default: 1e-4 Tolerance of the stopping condition. max_iter : integer, default: 200 Maximum number of iterations before timing out. random_state : int, RandomState instance or None, optional, default: None If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. alpha : double, default: 0. Constant that multiplies the regularization terms. Set it to zero to have no regularization. .. versionadded:: 0.17 *alpha* used in the Coordinate Descent solver. l1_ratio : double, default: 0. The regularization mixing parameter, with 0 <= l1_ratio <= 1. For l1_ratio = 0 the penalty is an elementwise L2 penalty (aka Frobenius Norm). For l1_ratio = 1 it is an elementwise L1 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2. .. versionadded:: 0.17 Regularization parameter *l1_ratio* used in the Coordinate Descent solver. verbose : bool, default=False Whether to be verbose. shuffle : boolean, default: False If true, randomize the order of coordinates in the CD solver. .. versionadded:: 0.17 *shuffle* parameter used in the Coordinate Descent solver. Attributes ---------- components_ : array, [n_components, n_features] Factorization matrix, sometimes called 'dictionary'. reconstruction_err_ : number Frobenius norm of the matrix difference, or beta-divergence, between the training data ``X`` and the reconstructed data ``WH`` from the fitted model. n_iter_ : int Actual number of iterations. Examples -------- >>> import numpy as np >>> X = np.array([[1, 1], [2, 1], [3, 1.2], [4, 1], [5, 0.8], [6, 1]]) >>> from sklearn.decomposition import NMF >>> model = NMF(n_components=2, init='random', random_state=0) >>> W = model.fit_transform(X) >>> H = model.components_ References ---------- Cichocki, Andrzej, and P. H. A. N. Anh-Huy. "Fast local algorithms for large scale nonnegative matrix and tensor factorizations." IEICE transactions on fundamentals of electronics, communications and computer sciences 92.3: 708-721, 2009. Fevotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Computation, 23(9). RQRSg-C6?igic C sg||_||_||_||_||_||_||_||_| |_| |_ | |_ dS(N( RpR[RYRZRRRfRGRHRR( tselfRpR[RYRZRRRfRGRHRR((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pyt__init__s          c!C st|dddt}td|d|d|d|jd |jd td |jd |jd |jd|j d|j d|j ddd|j d|j d|j\}}}t||||jdt|_|jd|_||_||_|S(sLearn a NMF model for the data X and returns the transformed data. This is more efficient than calling fit followed by transform. Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) Data matrix to be decomposed y : Ignored W : array-like, shape (n_samples, n_components) If init='custom', it is used as initial guess for the solution. H : array-like, shape (n_components, n_features) If init='custom', it is used as initial guess for the solution. Returns ------- W : array, shape (n_samples, n_components) Transformed data. RRRRRR*R+RpR[RRYRZRRRGRHRIRCRfRRR-i(RR(RRFRRpR[RRYRZRRRGRHRfRRR;treconstruction_err_Rt n_components_t components_tn_iter_(RRRzR*R+R((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pyt fit_transforms$   cK s|j|||S(sLearn a NMF model for the data X. Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) Data matrix to be decomposed y : Ignored Returns ------- self (R(RRRztparams((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pytfitsc!C st|dtd|ddd|jd|jd|jdtd|jd |jd |j d |j d |j d |j ddd|j d|jd|j\}}}|S(sUTransform the data X according to the fitted NMF model Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) Data matrix to be transformed by the model Returns ------- W : array, shape (n_samples, n_components) Transformed data RRR*R+RpR[RRYRZRRRGRHRIRCRfRRN(R RRURRR[tFalseRYRZRRRGRHRfRR(RRR*t_R((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pyt transforms cC s t|dtj||jS(syTransform data back to its original space. Parameters ---------- W : {array-like, sparse matrix}, shape (n_samples, n_components) Transformed data matrix Returns ------- X : {array-like, sparse matrix}, shape (n_samples, n_features) Data matrix of original shape .. versionadded:: 0.18 R(R RRR(RR*((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pytinverse_transforms N( t__name__t __module__t__doc__RURRRRRR(((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pyRs    *  (:Rt __future__RRtmathRRVRaRtnumpyRt scipy.sparsetsparseRtbaseRRtutilsRRt utils.extmathRRR R tutils.validationR R t exceptionsR t cdnmf_fastRRtfloat32RqR%tIntegraltintegerRRRRRR;R$RPR^RRURRRRRRRRR(((s8lib/python2.7/site-packages/sklearn/decomposition/nmf.pytsV     j      i^ R