\c@sdZddlZddlmZddlmZmZddlm Z ddl m Z dd l m Z dd lmZd Zd Zd ZdZdZdZdZdZdZdZdZdZdZdZdefdYZdS(sGaussian Mixture Model.iN(tlinalgi(t BaseMixturet _check_shapei(tzip(t check_array(tcheck_is_fitted(t row_normscCst|dtjtjgdt}t||fdttj|dsgttj|drt dtj |tj |fntj tj dtj|dst dtj|n|S(s'Check the user provided 'weights'. Parameters ---------- weights : array-like, shape (n_components,) The proportions of components of each mixture. n_components : int Number of components. Returns ------- weights : array, shape (n_components,) tdtypet ensure_2dtweightsgg?s]The parameter 'weights' should be in the range [0, 1], but got max value %.5f, min value %.5fsIThe parameter 'weights' should be normalized, but got sum(weights) = %.5f(Rtnptfloat64tfloat32tFalseRtanytlesstgreatert ValueErrortmintmaxtallclosetabstsum(R t n_components((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_check_weightss %(cCs>t|dtjtjgdt}t|||fd|S(sYValidate the provided 'means'. Parameters ---------- means : array-like, shape (n_components, n_features) The centers of the current components. n_components : int Number of components. n_features : int Number of features. Returns ------- means : array, (n_components, n_features) RRtmeans(RR R R R R(RRt n_features((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt _check_means6s$cCs2tjtj|dr.td|ndS(s.Check a precision vector is positive-definite.gs!'%s precision' should be positiveN(R Rt less_equalR(t precisiontcovariance_type((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_check_precision_positivityMscCsJtj||jo0tjtj|dksFtd|ndS(s<Check a precision matrix is symmetric and positive-definite.gs5'%s precision' should be symmetric, positive-definiteN(R RtTtallRteigvalshR(RR((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_check_precision_matrixTscCs"x|D]}t||qWdS(sACheck the precision matrices are symmetric and positive-definite.N(R#(t precisionsRtprec((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_check_precisions_full\s c Cst|dtjtjgdtd|dk}i|||fd6||fd6||fd6|fd6}t|||d|itd6td6td6td6}|||||S( sValidate user provided precisions. Parameters ---------- precisions : array-like 'full' : shape of (n_components, n_features, n_features) 'tied' : shape of (n_features, n_features) 'diag' : shape of (n_components, n_features) 'spherical' : shape of (n_components,) covariance_type : string n_components : int Number of components. n_features : int Number of features. Returns ------- precisions : array RRtallow_ndtfullttiedtdiagt sphericals %s precision( RR R R R RR&R#R(R$RRRtprecisions_shapet_check_precisions((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyR-bs       c Cs|j\}}tj|||f}x{t|D]m}|||} tj|dd|f| j| ||||<||jdd|dc|7|j\}}ytj|dt} Wn tjk rt|nXtj | tj |dtj }n=tj tj |drht|ndtj|}|S(s Compute the Cholesky decomposition of the precisions. Parameters ---------- covariances : array-like The covariance matrix of the current components. The shape depends of the covariance_type. covariance_type : {'full', 'tied', 'diag', 'spherical'} The type of precision matrices. Returns ------- precisions_cholesky : array-like The cholesky decomposition of sample precisions of the current components. The shape depends of the covariance_type. sFitting the mixture model failed because some components have ill-defined empirical covariance (for instance caused by singleton or collapsed samples). Try to decrease the number of components, or increase reg_covar.R(tlowerR)gg?(R.R R/t enumerateRtcholeskytTruet LinAlgErrorRtsolve_triangularteyeR RRtsqrt( R7Rt estimate_precision_error_messageRRt_tprecisions_cholR8R>tcov_chol((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_compute_precision_cholesky!s0   cCs|dkre|j\}}}tjtj|j|ddddd|dfd}np|dkrtjtjtj|}n@|dkrtjtj|dd}n|tj|}|S(sCompute the log-det of the cholesky decomposition of matrices. Parameters ---------- matrix_chol : array-like Cholesky decompositions of the matrices. 'full' : shape of (n_components, n_features, n_features) 'tied' : shape of (n_features, n_features) 'diag' : shape of (n_components, n_features) 'spherical' : shape of (n_components,) covariance_type : {'full', 'tied', 'diag', 'spherical'} n_features : int Number of features. Returns ------- log_det_precision_chol : array-like, shape (n_components,) The determinant of the precision matrix for each component. R(iNiR)R*RE(R.R RtlogtreshapeR*(t matrix_cholRRRRRt log_det_chol((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_compute_log_det_choleskyUs  5 $ !cCsT|j\}}|j\}}t|||}|dkrtj||f} xtt||D]b\} \} } tj|| tj| | } tjtj| dd| dd| f`. .. versionadded:: 0.18 Parameters ---------- n_components : int, defaults to 1. The number of mixture components. covariance_type : {'full' (default), 'tied', 'diag', 'spherical'} String describing the type of covariance parameters to use. Must be one of: 'full' each component has its own general covariance matrix 'tied' all components share the same general covariance matrix 'diag' each component has its own diagonal covariance matrix 'spherical' each component has its own single variance tol : float, defaults to 1e-3. The convergence threshold. EM iterations will stop when the lower bound average gain is below this threshold. reg_covar : float, defaults to 1e-6. Non-negative regularization added to the diagonal of covariance. Allows to assure that the covariance matrices are all positive. max_iter : int, defaults to 100. The number of EM iterations to perform. n_init : int, defaults to 1. The number of initializations to perform. The best results are kept. init_params : {'kmeans', 'random'}, defaults to 'kmeans'. The method used to initialize the weights, the means and the precisions. Must be one of:: 'kmeans' : responsibilities are initialized using kmeans. 'random' : responsibilities are initialized randomly. weights_init : array-like, shape (n_components, ), optional The user-provided initial weights, defaults to None. If it None, weights are initialized using the `init_params` method. means_init : array-like, shape (n_components, n_features), optional The user-provided initial means, defaults to None, If it None, means are initialized using the `init_params` method. precisions_init : array-like, optional. The user-provided initial precisions (inverse of the covariance matrices), defaults to None. If it None, precisions are initialized using the 'init_params' method. The shape depends on 'covariance_type':: (n_components,) if 'spherical', (n_features, n_features) if 'tied', (n_components, n_features) if 'diag', (n_components, n_features, n_features) if 'full' random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. warm_start : bool, default to False. If 'warm_start' is True, the solution of the last fitting is used as initialization for the next call of fit(). This can speed up convergence when fit is called several times on similar problems. In that case, 'n_init' is ignored and only a single initialization occurs upon the first call. See :term:`the Glossary `. verbose : int, default to 0. Enable verbose output. If 1 then it prints the current initialization and each iteration step. If greater than 1 then it prints also the log probability and the time needed for each step. verbose_interval : int, default to 10. Number of iteration done before the next print. Attributes ---------- weights_ : array-like, shape (n_components,) The weights of each mixture components. means_ : array-like, shape (n_components, n_features) The mean of each mixture component. covariances_ : array-like The covariance of each mixture component. The shape depends on `covariance_type`:: (n_components,) if 'spherical', (n_features, n_features) if 'tied', (n_components, n_features) if 'diag', (n_components, n_features, n_features) if 'full' precisions_ : array-like The precision matrices for each component in the mixture. A precision matrix is the inverse of a covariance matrix. A covariance matrix is symmetric positive definite so the mixture of Gaussian can be equivalently parameterized by the precision matrices. Storing the precision matrices instead of the covariance matrices makes it more efficient to compute the log-likelihood of new samples at test time. The shape depends on `covariance_type`:: (n_components,) if 'spherical', (n_features, n_features) if 'tied', (n_components, n_features) if 'diag', (n_components, n_features, n_features) if 'full' precisions_cholesky_ : array-like The cholesky decomposition of the precision matrices of each mixture component. A precision matrix is the inverse of a covariance matrix. A covariance matrix is symmetric positive definite so the mixture of Gaussian can be equivalently parameterized by the precision matrices. Storing the precision matrices instead of the covariance matrices makes it more efficient to compute the log-likelihood of new samples at test time. The shape depends on `covariance_type`:: (n_components,) if 'spherical', (n_features, n_features) if 'tied', (n_components, n_features) if 'diag', (n_components, n_features, n_features) if 'full' converged_ : bool True when convergence was reached in fit(), False otherwise. n_iter_ : int Number of step used by the best fit of EM to reach the convergence. lower_bound_ : float Lower bound value on the log-likelihood (of the training data with respect to the model) of the best fit of EM. See Also -------- BayesianGaussianMixture : Gaussian mixture model fit with a variational inference. iR(gMbP?gư>idtkmeansii cCswtt|jd|d|d|d|d|d|d| d| d | d | ||_||_| |_| |_dS( NRttolR6tmax_itertn_initt init_paramst random_statet warm_starttverbosetverbose_interval(tsuperRft__init__Rt weights_initt means_inittprecisions_init(tselfRRRhR6RiRjRkRrRsRtRlRmRnRo((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyRqMs     cCs|j\}}|jdkr4td|jn|jdk r^t|j|j|_n|jdk rt|j|j||_n|j dk rt |j |j|j||_ ndS(s7Check the Gaussian mixture parameters are well defined.R+R)R*R(slInvalid value for 'covariance_type': %s 'covariance_type' should be in ['spherical', 'tied', 'diag', 'full']N(R+R)R*R(( R.RRRrtNoneRRRsRRtR-(RuR4RRR((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_check_parameters]s   c Cs5|j\}}t|||j|j\}}}||:}|jdkrR|n|j|_|jdkrs|n|j|_|j dkr||_ t ||j|_ n|jdkrt jg|j D]}tj|dt^q|_ n9|jdkr%tj|j dt|_ n |j |_ dS(sInitialization of the Gaussian mixture parameters. Parameters ---------- X : array-like, shape (n_samples, n_features) resp : array-like, shape (n_samples, n_components) R(RIR)N(R.RHR6RRrRvtweights_Rstmeans_Rtt covariances_RUtprecisions_cholesky_R tarrayRRKRL( RuR4R3R_RRR RR7t prec_init((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt _initializets& !  ! 1cCsp|j\}}t|tj||j|j\|_|_|_|j|:_t |j|j|_ dS(s&M step. Parameters ---------- X : array-like, shape (n_samples, n_features) log_resp : array-like, shape (n_samples, n_components) Logarithm of the posterior probabilities (or responsibilities) of the point of each sample in X. N( R.RHR texpR6RRxRyRzRUR{(RuR4tlog_respR_RR((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_m_steps cCst||j|j|jS(N(ReRyR{R(RuR4((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_estimate_log_probscCstj|jS(N(R RVRx(Ru((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_estimate_log_weightsscCs|S(N((RuRRt log_prob_norm((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_compute_lower_boundscCst|dddgdS(NRxRyR{(R(Ru((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_check_is_fittedscCs|j|j|j|jfS(N(RxRyRzR{(Ru((s?lib/python2.7/site-packages/sklearn/mixture/gaussian_mixture.pyt_get_parametersscCs|\|_|_|_|_|jj\}}|jdkrtj|jj|_x|t |jD](\}}tj ||j |j|s,  !     -     ' 4 ( 6