B \DG@sdZddlmZddlZddlmZmZddlmZddlZ ddl m Z ddl m Z dd l mZdd lmZdd lmZdd lmZmZdd lmZddZdddZGdddeeee ZdS)zBase class for mixture models.)print_functionN)ABCMetaabstractmethod)time)cluster) BaseEstimator) DensityMixin)six)ConvergenceWarning) check_arraycheck_random_state) logsumexpcCs,t|}|j|kr(td|||jfdS)zValidate the shape of the input parameter 'param'. Parameters ---------- param : array param_shape : tuple name : string z:The parameter '%s' should have the shape of %s, but got %sN)npZarrayshape ValueError)ZparamZ param_shapenamer3lib/python3.7/site-packages/sklearn/mixture/base.py _check_shapes  rcCsrt|tjtjg|d}|dk rB|jd|krBtd||jdf|dk rn|jd|krntd||jdf|S)zCheck the input data X. Parameters ---------- X : array-like, shape (n_samples, n_features) n_components : int Returns ------- X : array, shape (n_samples, n_features) )dtypeensure_min_samplesNrzLExpected n_samples >= n_components but got n_components = %d, n_samples = %drz?Expected the input data X have %d features, but got %d features)r rZfloat64Zfloat32rr)X n_components n_featuresrrrr_check_X)s rc@seZdZdZddZddZeddZdd Zed d Z d4d dZ d5ddZ ddZ eddZ eddZeddZeddZddZd6ddZdd Zd!d"Zd7d$d%Zd&d'Zed(d)Zed*d+Zd,d-Zd.d/Zd0d1Zd2d3Zd S)8 BaseMixturezBase class for mixture models. This abstract class specifies an interface for all mixture classes and provides basic common methods for mixture models. c Cs@||_||_||_||_||_||_||_||_| |_| |_ dS)N) rtol reg_covarmax_itern_init init_params random_state warm_startverboseverbose_interval) selfrrrr r!r"r#r$r%r&rrr__init__JszBaseMixture.__init__cCs|jdkrtd|j|jdkr0td|j|jdkrHtd|j|jdkr`td|j|jdkrxtd|j||dS) zCheck values of the basic parameters. Parameters ---------- X : array-like, shape (n_samples, n_features) rzOInvalid value for 'n_components': %d Estimation requires at least one componentgzKInvalid value for 'tol': %.5f Tolerance used by the EM must be non-negativezCInvalid value for 'n_init': %d Estimation requires at least one runzKInvalid value for 'max_iter': %d Estimation requires at least one iterationzUInvalid value for 'reg_covar': %.5f regularization on covariance must be non-negativeN)rrrr!r r_check_parameters)r'rrrr_check_initial_parametersXs           z%BaseMixture._check_initial_parameterscCsdS)zCheck initial parameters of the derived class. Parameters ---------- X : array-like, shape (n_samples, n_features) Nr)r'rrrrr)|szBaseMixture._check_parameterscCs|j\}}|jdkrRt||jf}tj|jd|d|j}d|t ||f<nF|jdkr| ||j}||j ddddtj f}nt d|j|||dS)zInitialize the model parameters. Parameters ---------- X : array-like, shape (n_samples, n_features) random_state : RandomState A random number generator instance. Zkmeansr)Z n_clustersr!r#Zrandom)axisNz(Unimplemented initialization method '%s')rr"rZzerosrrZKMeansfitZlabels_ZarangeZrandsumnewaxisr _initialize)r'rr# n_samples_respZlabelrrr_initialize_parameterss      z"BaseMixture._initialize_parameterscCsdS)zInitialize the model parameters of the derived class. Parameters ---------- X : array-like, shape (n_samples, n_features) resp : array-like, shape (n_samples, n_components) Nr)r'rr2rrrr/s zBaseMixture._initializeNcCs||||S)awEstimate model parameters with the EM algorithm. The method fits the model ``n_init`` times and sets the parameters with which the model has the largest likelihood or lower bound. Within each trial, the method iterates between E-step and M-step for ``max_iter`` times until the change of likelihood or lower bound is less than ``tol``, otherwise, a ``ConvergenceWarning`` is raised. If ``warm_start`` is ``True``, then ``n_init`` is ignored and a single initialization is performed upon the first call. Upon consecutive calls, training starts where it left off. Parameters ---------- X : array-like, shape (n_samples, n_features) List of n_features-dimensional data points. Each row corresponds to a single data point. Returns ------- self ) fit_predict)r'ryrrrr,s zBaseMixture.fitcCsvt||jdd}|||jo(t|d }|r6|jnd}tj }d|_t |j }|j \}}xt |D]} | | |r||||rtj n|j} xlt d|jdD]X} | } ||\} }|||||| } | | }|| |t||jkrd|_PqW|| | |krf| }|}| }qfW||\}}|jsTtd| dt||||_||_|jddS) acEstimate model parameters using X and predict the labels for X. The method fits the model n_init times and sets the parameters with which the model has the largest likelihood or lower bound. Within each trial, the method iterates between E-step and M-step for `max_iter` times until the change of likelihood or lower bound is less than `tol`, otherwise, a `ConvergenceWarning` is raised. After fitting, it predicts the most probable label for the input data points. .. versionadded:: 0.20 Parameters ---------- X : array-like, shape (n_samples, n_features) List of n_features-dimensional data points. Each row corresponds to a single data point. Returns ------- labels : array, shape (n_samples,) Component labels. r)r converged_rFTzzInitialization %d did not converge. Try different init parameters, or increase max_iter, tol or check for degenerate data.)r+)rrr*r$hasattrr!rZinftyr6r r#rrange_print_verbose_msg_init_begr3Z lower_bound_r _e_step_m_stepZ_compute_lower_bound_print_verbose_msg_iter_endabsr_print_verbose_msg_init_end_get_parameterswarningswarnr _set_parametersZn_iter_argmax)r'rr5Zdo_initr!Zmax_lower_boundr#r0r1ZinitZ lower_boundn_iterZprev_lower_bound log_prob_normlog_respZchangeZ best_paramsZ best_n_iterrrrr4sJ         zBaseMixture.fit_predictcCs||\}}t||fS)aE step. Parameters ---------- X : array-like, shape (n_samples, n_features) Returns ------- log_prob_norm : float Mean of the logarithms of the probabilities of each sample in X log_responsibility : array, shape (n_samples, n_components) Logarithm of the posterior probabilities (or responsibilities) of the point of each sample in X. )_estimate_log_prob_resprmean)r'rrErFrrrr:szBaseMixture._e_stepcCsdS)a&M step. Parameters ---------- X : array-like, shape (n_samples, n_features) log_resp : array-like, shape (n_samples, n_components) Logarithm of the posterior probabilities (or responsibilities) of the point of each sample in X. Nr)r'rrFrrrr;,s zBaseMixture._m_stepcCsdS)Nr)r'rrr_check_is_fitted:szBaseMixture._check_is_fittedcCsdS)Nr)r'rrrr?>szBaseMixture._get_parameterscCsdS)Nr)r'ZparamsrrrrBBszBaseMixture._set_parameterscCs.|t|d|jjd}t||ddS)aCompute the weighted log probabilities for each sample. Parameters ---------- X : array-like, shape (n_samples, n_features) List of n_features-dimensional data points. Each row corresponds to a single data point. Returns ------- log_prob : array, shape (n_samples,) Log probabilities of each data point in X. Nr)r+)rIrmeans_rr_estimate_weighted_log_prob)r'rrrr score_samplesFszBaseMixture.score_samplescCs||S)aCompute the per-sample average log-likelihood of the given data X. Parameters ---------- X : array-like, shape (n_samples, n_dimensions) List of n_features-dimensional data points. Each row corresponds to a single data point. Returns ------- log_likelihood : float Log likelihood of the Gaussian mixture given X. )rLrH)r'rr5rrrscoreYszBaseMixture.scorecCs.|t|d|jjd}||jddS)aPredict the labels for the data samples in X using trained model. Parameters ---------- X : array-like, shape (n_samples, n_features) List of n_features-dimensional data points. Each row corresponds to a single data point. Returns ------- labels : array, shape (n_samples,) Component labels. Nr)r+)rIrrJrrKrC)r'rrrrpredictiszBaseMixture.predictcCs4|t|d|jjd}||\}}t|S)aPredict posterior probability of each component given the data. Parameters ---------- X : array-like, shape (n_samples, n_features) List of n_features-dimensional data points. Each row corresponds to a single data point. Returns ------- resp : array, shape (n_samples, n_components) Returns the probability each Gaussian (state) in the model given each sample. Nr)rIrrJrrGrZexp)r'rr1rFrrr predict_proba{szBaseMixture.predict_probarcs|dkrtdjjj\}tj|j}j dkrrt fddt jj |D}nTj dkrt fddt j|D}n&t fddt jj |D}t d dt|D}||fS) aGenerate random samples from the fitted Gaussian distribution. Parameters ---------- n_samples : int, optional Number of samples to generate. Defaults to 1. Returns ------- X : array, shape (n_samples, n_features) Randomly generated sample y : array, shape (nsamples,) Component labels rzNInvalid value for 'n_samples': %d . The sampling requires at least one sample.fullcs$g|]\}}}||t|qSr)multivariate_normalint).0rH covariancesample)rngrr sz&BaseMixture.sample..Ztiedcs$g|]\}}|jt|qSr)rQ covariances_rR)rSrHrU)rVr'rrrWscs,g|]$\}}}||t|qSr)ZrandnrZsqrt)rSrHrTrU)rrVrrrWscSs g|]\}}tj||tdqS))r)rrPrR)rSjrUrrrrWs)rIrrrJrr r#Z multinomialZweights_Zcovariance_typerZvstackziprXZ concatenate enumerate)r'r0r1Zn_samples_comprr5r)rrVr'rrUs2         zBaseMixture.samplecCs|||S)a Estimate the weighted log-probabilities, log P(X | Z) + log weights. Parameters ---------- X : array-like, shape (n_samples, n_features) Returns ------- weighted_log_prob : array, shape (n_samples, n_component) )_estimate_log_prob_estimate_log_weights)r'rrrrrKs z'BaseMixture._estimate_weighted_log_probcCsdS)zEstimate log-weights in EM algorithm, E[ log pi ] in VB algorithm. Returns ------- log_weight : array, shape (n_components, ) Nr)r'rrrr]sz!BaseMixture._estimate_log_weightscCsdS)a7Estimate the log-probabilities log P(X | Z). Compute the log-probabilities per each component for each sample. Parameters ---------- X : array-like, shape (n_samples, n_features) Returns ------- log_prob : array, shape (n_samples, n_component) Nr)r'rrrrr\szBaseMixture._estimate_log_probc CsL||}t|dd}tjdd||ddtjf}WdQRX||fS)a>Estimate log probabilities and responsibilities for each sample. Compute the log probabilities, weighted log probabilities per component and responsibilities for each sample in X with respect to the current state of the model. Parameters ---------- X : array-like, shape (n_samples, n_features) Returns ------- log_prob_norm : array, shape (n_samples,) log p(X) log_responsibilities : array, shape (n_samples, n_components) logarithm of the responsibilities r)r+ignore)ZunderN)rKrrZerrstater.)r'rZweighted_log_probrErFrrrrGs    z#BaseMixture._estimate_log_prob_respcCsB|jdkrtd|n&|jdkr>td|t|_|j|_dS)z(Print verbose message on initialization.rzInitialization %drN)r%printr_init_prev_time_iter_prev_time)r'r!rrrr9s    z'BaseMixture._print_verbose_msg_init_begcCsX||jdkrT|jdkr&td|n.|jdkrTt}td|||j|f||_dS)z(Print verbose message on initialization.rrz Iteration %drz0 Iteration %d time lapse %.5fs ll change %.5fN)r&r%r_rra)r'rDZdiff_llZcur_timerrrr< s  z'BaseMixture._print_verbose_msg_iter_endcCsD|jdkrtd|jn&|jdkr@td|jt|j|fdS)z.Print verbose message on the end of iteration.rzInitialization converged: %srz7Initialization converged: %s time lapse %.5fs ll %.5fN)r%r_r6rr`)r'Zllrrrr>s   z'BaseMixture._print_verbose_msg_init_end)N)N)N)r)__name__ __module__ __qualname____doc__r(r*rr)r3r/r,r4r:r;rIr?rBrLrMrNrOrUrKr]r\rGr9r<r>rrrrrCs2$   T      1   r)NNr)reZ __future__rr@abcrrrZnumpyrrbaserr Z externalsr exceptionsr Zutilsr r Z utils.fixesrrrZwith_metaclassrrrrrs