\c @ sdZddlmZddlZddlZddljZddl m Z m Z m Z ddl mZddl mZddlmZmZmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddl m!Z!ddl m"Z"ddl m#Z#ddl$m%Z%ddl&m'Z'ddl(m)Z)ddl*m+Z+e,dZ-dZ.dZ/dZ0e,dddd e1d!e,e2e,de1d" Z3d de1e,e,d!e2d#Z4d de1e,e,d!e2d$Z5d%Z6e2e,d&Z7e,e,e,d'Z8d(e e e fd)YZ9e1e,d*e1d+Z:d,d-Z;d.e9fd/YZ<dS(0sK-means clusteringi(tdivisionNi(t BaseEstimatort ClusterMixintTransformerMixin(teuclidean_distances(tpairwise_distances_argmin_min(t row_normst squared_normt stable_cumsum(tassign_rows_csr(tmean_variance_axis(t _num_samples(t check_array(t gen_batches(tcheck_random_state(tcheck_is_fitted(t FLOAT_DTYPES(tParallel(tdelayed(teffective_n_jobs(t string_types(tConvergenceWarningi(t_k_means(t k_means_elkancC s|j\}}tj||fd|j}|dk sEtd|dkrmdttj|}n|j|}t j |r||j |d`. n_local_trials : integer, optional The number of seeding trials for each center (except the first), of which the one reducing inertia the most is greedily chosen. Set to None to make the number of trials depend logarithmically on the number of seeds (2+log(k)); this is the default. Notes ----- Selects initial cluster centers for k-mean clustering in a smart way to speed up convergence. see: Arthur, D. and Vassilvitskii, S. "k-means++: the advantages of careful seeding". ACM-SIAM symposium on Discrete algorithms. 2007 Version ported from http://www.stanford.edu/~darthur/kMeansppTest.zip, which is the implementation used in the aforementioned paper. tdtypesx_squared_norms None in _k_initiitY_norm_squaredtsquarediN(tshapetnptemptyRtNonetAssertionErrortinttlogtrandinttsptissparsettoarrayRtnewaxistTruetsumtranget random_samplet searchsortedRtminimum(tXt n_clusterstx_squared_normst random_statetn_local_trialst n_samplest n_featurestcenterst center_idtclosest_dist_sqt current_pottct rand_valst candidate_idstdistance_to_candidatestbest_candidatetbest_pott best_dist_sqttrialt new_dist_sqtnew_pot((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyt_k_init-sH$          cC sst||kr.td|j|fn|jd|jdkrotd|jd|jdfndS(s3Check if centers is compatible with X and n_centerssNThe shape of the initial centers (%s) does not match the number of clusters %iisfThe number of features of the initial centers %s does not match the number of features of the data %s.N(tlent ValueErrorR(R-t n_centersR4((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyt_validate_center_shapescC sNtj|r(t|ddd}ntj|dd}tj||S(s6Return a tolerance which is independent of the datasettaxisii(R#R$R Rtvartmean(R-ttolt variances((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyt _tolerancescC s|jd}|dkr/tj|d|jStj|}|t|krotd|t|fn||j}||j |jSdS(s6Set sample_weight if None, and check for correct dtypeiRs/n_samples=%d should be == len(sample_weight)=%dN( RRRtonesRtasarrayRCRDR(tastype(R-t sample_weightR2tscale((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyt_check_sample_weights  s k-means++tautoi i,g-C6?c sx|dkrtd|nt| } dkrJtdn| rVdnd#}tdddtjtjgd|d | tkrtd tfntd krj d}|d kn"t t r ntd t drtdj jd tt|dkrtjd|tddd}qntjsjdd}|8t dr|8qntdt d$\}}}dkr d} n| d kr3tjr*dnd} n| dkrHtn+| dkr]tntdt| t| dkr&xWt|D]}dddddd d| \}}}}|d#ks||kr|j}|j}|}|}qqWn| jtjtj j!d|}t"d | dd f d!|D}t#|\}}}}tj$|}||}||}||}||}tjs| s|7n||7}nt%t&|}|krQtjd"j'|t(ddn| rg||||fS|||fSd#S(%s5K-means clustering algorithm. Read more in the :ref:`User Guide `. Parameters ---------- X : array-like or sparse matrix, shape (n_samples, n_features) The observations to cluster. It must be noted that the data will be converted to C ordering, which will cause a memory copy if the given data is not C-contiguous. n_clusters : int The number of clusters to form as well as the number of centroids to generate. sample_weight : array-like, shape (n_samples,), optional The weights for each observation in X. If None, all observations are assigned equal weight (default: None) init : {'k-means++', 'random', or ndarray, or a callable}, optional Method for initialization, default to 'k-means++': 'k-means++' : selects initial cluster centers for k-mean clustering in a smart way to speed up convergence. See section Notes in k_init for more details. 'random': choose k observations (rows) at random from data for the initial centroids. If an ndarray is passed, it should be of shape (n_clusters, n_features) and gives the initial centers. If a callable is passed, it should take arguments X, k and and a random state and return an initialization. precompute_distances : {'auto', True, False} Precompute distances (faster but takes more memory). 'auto' : do not precompute distances if n_samples * n_clusters > 12 million. This corresponds to about 100MB overhead per job using double precision. True : always precompute distances False : never precompute distances n_init : int, optional, default: 10 Number of time the k-means algorithm will be run with different centroid seeds. The final results will be the best output of n_init consecutive runs in terms of inertia. max_iter : int, optional, default 300 Maximum number of iterations of the k-means algorithm to run. verbose : boolean, optional Verbosity mode. tol : float, optional The relative increment in the results before declaring convergence. random_state : int, RandomState instance or None (default) Determines random number generation for centroid initialization. Use an int to make the randomness deterministic. See :term:`Glossary `. copy_x : boolean, optional When pre-computing distances it is more numerically accurate to center the data first. If copy_x is True (default), then the original data is not modified, ensuring X is C-contiguous. If False, the original data is modified, and put back before the function returns, but small numerical differences may be introduced by subtracting and then adding the data mean, in this case it will also not ensure that data is C-contiguous which may cause a significant slowdown. n_jobs : int or None, optional (default=None) The number of jobs to use for the computation. This works by computing each of the n_init runs in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. algorithm : "auto", "full" or "elkan", default="auto" K-means algorithm to use. The classical EM-style algorithm is "full". The "elkan" variation is more efficient by using the triangle inequality, but currently doesn't support sparse data. "auto" chooses "elkan" for dense data and "full" for sparse data. return_n_iter : bool, optional Whether or not to return the number of iterations. Returns ------- centroid : float ndarray with shape (k, n_features) Centroids found at the last iteration of k-means. label : integer ndarray with shape (n_samples,) label[i] is the code or index of the centroid the i'th observation is closest to. inertia : float The final value of the inertia criterion (sum of squared distances to the closest centroid for all observations in the training set). best_n_iter : int Number of iterations corresponding to the best results. Returned only if `return_n_iter` is set to True. isFInvalid number of initializations. n_init=%d must be bigger than zero.s@Number of iterations should be a positive number, got %d insteadtCt accept_sparsetcsrRtordertcopys'n_samples=%d should be >= n_clusters=%dRSg`fAsQprecompute_distances should be 'auto' or True/False, but a value of %r was passedt __array__isaExplicit initial center position passed: performing only one init in k-means instead of n_init=%dt stackleveliRGRtfulltelkans3Algorithm must be 'auto', 'full' or 'elkan', got %stmax_itertinittverbosetprecompute_distancesRJR/R0tsizetn_jobsc3 sQ|]G}tdddddd d|VqdS(R]R^R_RJR`R/R0N(R(t.0tseed( R-R^t kmeans_singleR]R.R`RPRJR_R/(s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pys sskNumber of distinct clusters ({}) found smaller than n_clusters ({}). Possibly due to duplicate points in X.N(NNN()RDRRR Rtfloat64tfloat32R RLRt isinstancetboolthasattrRttypeR'RFtwarningstwarntRuntimeWarningR#R$RIRt_kmeans_single_lloydt_kmeans_single_elkantstrRR)RXR"tiinfotint32tmaxRtziptargminRCtsettformatR(R-R.RPR^R`tn_initR]R_RJR0tcopy_xRbt algorithmt return_n_iterRWR2tX_meant best_labelst best_inertiat best_centerstittlabelstinertiaR4tn_iter_t best_n_itertseedstresultstn_iterstbesttdistinct_clusters(( R-R^ReR]R.R`RPRJR_R/s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pytk_meanssq     !                 $'          c  C sNtj|rtdnt|}|dkrKt|dt}nt|||d|d|} tj | } |rdGHnt ||} t || || d|d|d|\} } } |dkrtj || | d d tj }nFtj || | d d d d tj | }tj |d tj }| || | fS( Ns2algorithm='elkan' not supported for sparse input XRR0R/sInitialization completeRJR]R_iRRGi(R#R$t TypeErrorRRRR't_init_centroidsRtascontiguousarrayRRRR(Rf(R-RPR.R]R^R_R/R0RJR`R4tchecked_sample_weightRtn_iterRt sq_distances((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyRps(     'c  C st|}t||}d \} } } t|||d|d|} |rVdGHntjd|jdfd|j}xt|D] }| j }t |||| d| d|\}}t j |rt j|||||} nt j|||||} |r!d ||fGHn| d ks9|| krZ|j } | j } |} nt|| }||kr|rd |||fGHnPqqW|dkrt |||| d| d|\} } n| | | |d fS(sA single run of k-means, assumes preparation completed prior. Parameters ---------- X : array-like of floats, shape (n_samples, n_features) The observations to cluster. n_clusters : int The number of clusters to form as well as the number of centroids to generate. sample_weight : array-like, shape (n_samples,) The weights for each observation in X. max_iter : int, optional, default 300 Maximum number of iterations of the k-means algorithm to run. init : {'k-means++', 'random', or ndarray, or a callable}, optional Method for initialization, default to 'k-means++': 'k-means++' : selects initial cluster centers for k-mean clustering in a smart way to speed up convergence. See section Notes in k_init for more details. 'random': choose k observations (rows) at random from data for the initial centroids. If an ndarray is passed, it should be of shape (k, p) and gives the initial centers. If a callable is passed, it should take arguments X, k and and a random state and return an initialization. tol : float, optional The relative increment in the results before declaring convergence. verbose : boolean, optional Verbosity mode x_squared_norms : array Precomputed x_squared_norms. precompute_distances : boolean, default: True Precompute distances (faster but takes more memory). random_state : int, RandomState instance or None (default) Determines random number generation for centroid initialization. Use an int to make the randomness deterministic. See :term:`Glossary `. Returns ------- centroid : float ndarray with shape (k, n_features) Centroids found at the last iteration of k-means. label : integer ndarray with shape (n_samples,) label[i] is the code or index of the centroid the i'th observation is closest to. inertia : float The final value of the inertia criterion (sum of squared distances to the closest centroid for all observations in the training set). n_iter : int Number of iterations run. R0R/sInitialization completeRiRR`t distancessIteration %2d, inertia %.3fs>Converged at iteration %d: center shift %e within tolerance %eiN(NNN(RRRRRRtzerosRRR)RXt_labels_inertiaR#R$Rt_centers_sparset_centers_denseR(R-RPR.R]R^R_R/R0RJR`R~RRR4Rtit centers_oldRRtcenter_shift_total((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyRosFF  %       c C s|jd}td|d|ddditd6\}}|jtj}||jdkrj||(n||j}||fS(shCompute labels and inertia using a full distance matrix. This will overwrite the 'distances' array in-place. Parameters ---------- X : numpy array, shape (n_sample, n_features) Input data. sample_weight : array-like, shape (n_samples,) The weights for each observation in X. x_squared_norms : numpy array, shape (n_samples,) Precomputed squared norms of X. centers : numpy array, shape (n_clusters, n_features) Cluster centers which data is assigned to. distances : numpy array, shape (n_samples,) Pre-allocated array in which distances are stored. Returns ------- labels : numpy array, dtype=np.int, shape (n_samples,) Indices of clusters that samples are assigned to. inertia : float Sum of squared distances of samples to their closest cluster center. iR-tYtmetrict euclideant metric_kwargsR(RRR'RORRsR(( R-RPR/R4RR2RtmindistR((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyt _labels_inertia_precompute_denseGs ( c C s|jd}t||}tj|dtj}|dkr^tjddd|j}ntj |rt j |||||d|}n=|rt |||||St j |||||d|}||fS(sLE step of the K-means EM algorithm. Compute the labels and the inertia of the given samples and centers. This will compute the distances in-place. Parameters ---------- X : float64 array-like or CSR sparse matrix, shape (n_samples, n_features) The input samples to assign to the labels. sample_weight : array-like, shape (n_samples,) The weights for each observation in X. x_squared_norms : array, shape (n_samples,) Precomputed squared euclidean norm of each data point, to speed up computations. centers : float array, shape (k, n_features) The cluster centers. precompute_distances : boolean, default: True Precompute distances (faster but takes more memory). distances : float array, shape (n_samples,) Pre-allocated array to be filled in with each sample's distance to the closest center. Returns ------- labels : int array of shape(n) The resulting assignment inertia : float Sum of squared distances of samples to their closest cluster center. iiRRRN(i(RRRRR[RsRRRR#R$Rt_assign_labels_csrRt_assign_labels_array( R-RPR/R4R`RR2RR((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyRws"%     c C st|}|jd}|dkr:t|dt}n|dk r||kr||krtjd||ftddd|}n|jd||}||}||}|jd}n%||krt d||fnt |t r"|dkr"t ||d |d |}nt |t r]|d kr]|j || } || }nt|d rtj|d |j}nXt|r|||d |}tj|d |j}nt d|t|ftj|r|j}nt||||S(s+Compute the initial centroids Parameters ---------- X : array, shape (n_samples, n_features) k : int number of centroids init : {'k-means++', 'random' or ndarray or callable} optional Method for initialization random_state : int, RandomState instance or None (default) Determines random number generation for centroid initialization. Use an int to make the randomness deterministic. See :term:`Glossary `. x_squared_norms : array, shape (n_samples,), optional Squared euclidean norm of each data point. Pass it if you have it at hands already to avoid it being recomputed here. Default: None init_size : int, optional Number of samples to randomly sample for speeding up the initialization (sometimes at the expense of accuracy): the only algorithm is initialized by running a batch KMeans on a random subset of the data. This needs to be larger than k. Returns ------- centers : array, shape(k, n_features) iRs:init_size=%d should be larger than k=%d. Setting it to 3*kRZiis'n_samples=%d should be larger than k=%ds k-means++R0R/trandomRYRspthe init parameter for the k-means should be 'k-means++' or 'random' or an ndarray, '%s' (type '%s') was passed.N(RRRRR'RlRmRnR"RDRhRRBt permutationRjRtarrayRtcallableRNRkR#R$R%RF( R-tkR^R0R/t init_sizeR2t init_indicesR4R((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyRsF"             tKMeansc B seZdZddddddddeddd Zd Zddd Zddd Zddd Z d Z dZ ddZ dddZ RS(sK-Means clustering Read more in the :ref:`User Guide `. Parameters ---------- n_clusters : int, optional, default: 8 The number of clusters to form as well as the number of centroids to generate. init : {'k-means++', 'random' or an ndarray} Method for initialization, defaults to 'k-means++': 'k-means++' : selects initial cluster centers for k-mean clustering in a smart way to speed up convergence. See section Notes in k_init for more details. 'random': choose k observations (rows) at random from data for the initial centroids. If an ndarray is passed, it should be of shape (n_clusters, n_features) and gives the initial centers. n_init : int, default: 10 Number of time the k-means algorithm will be run with different centroid seeds. The final results will be the best output of n_init consecutive runs in terms of inertia. max_iter : int, default: 300 Maximum number of iterations of the k-means algorithm for a single run. tol : float, default: 1e-4 Relative tolerance with regards to inertia to declare convergence precompute_distances : {'auto', True, False} Precompute distances (faster but takes more memory). 'auto' : do not precompute distances if n_samples * n_clusters > 12 million. This corresponds to about 100MB overhead per job using double precision. True : always precompute distances False : never precompute distances verbose : int, default 0 Verbosity mode. random_state : int, RandomState instance or None (default) Determines random number generation for centroid initialization. Use an int to make the randomness deterministic. See :term:`Glossary `. copy_x : boolean, optional When pre-computing distances it is more numerically accurate to center the data first. If copy_x is True (default), then the original data is not modified, ensuring X is C-contiguous. If False, the original data is modified, and put back before the function returns, but small numerical differences may be introduced by subtracting and then adding the data mean, in this case it will also not ensure that data is C-contiguous which may cause a significant slowdown. n_jobs : int or None, optional (default=None) The number of jobs to use for the computation. This works by computing each of the n_init runs in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. algorithm : "auto", "full" or "elkan", default="auto" K-means algorithm to use. The classical EM-style algorithm is "full". The "elkan" variation is more efficient by using the triangle inequality, but currently doesn't support sparse data. "auto" chooses "elkan" for dense data and "full" for sparse data. Attributes ---------- cluster_centers_ : array, [n_clusters, n_features] Coordinates of cluster centers. If the algorithm stops before fully converging (see ``tol`` and ``max_iter``), these will not be consistent with ``labels_``. labels_ : Labels of each point inertia_ : float Sum of squared distances of samples to their closest cluster center. n_iter_ : int Number of iterations run. Examples -------- >>> from sklearn.cluster import KMeans >>> import numpy as np >>> X = np.array([[1, 2], [1, 4], [1, 0], ... [10, 2], [10, 4], [10, 0]]) >>> kmeans = KMeans(n_clusters=2, random_state=0).fit(X) >>> kmeans.labels_ array([1, 1, 1, 0, 0, 0], dtype=int32) >>> kmeans.predict([[0, 0], [12, 3]]) array([1, 0], dtype=int32) >>> kmeans.cluster_centers_ array([[10., 2.], [ 1., 2.]]) See also -------- MiniBatchKMeans Alternative online implementation that does incremental updates of the centers positions using mini-batches. For large scale learning (say n_samples > 10k) MiniBatchKMeans is probably much faster than the default batch implementation. Notes ------ The k-means problem is solved using either Lloyd's or Elkan's algorithm. The average complexity is given by O(k n T), were n is the number of samples and T is the number of iteration. The worst case complexity is given by O(n^(k+2/p)) with n = n_samples, p = n_features. (D. Arthur and S. Vassilvitskii, 'How slow is the k-means method?' SoCG2006) In practice, the k-means algorithm is very fast (one of the fastest clustering algorithms available), but it falls in local minima. That's why it can be useful to restart it several times. If the algorithm stops before fully converging (because of ``tol`` or ``max_iter``), ``labels_`` and ``cluster_centers_`` will not be consistent, i.e. the ``cluster_centers_`` will not be the means of the points in each cluster. Also, the estimator will reassign ``labels_`` after the last iteration to make ``labels_`` consistent with ``predict`` on the training set. is k-means++i i,g-C6?RSic C sg||_||_||_||_||_||_||_||_| |_| |_ | |_ dS(N( R.R^R]RJR`RyR_R0RzRbR{( tselfR.R^RyR]RJR`R_R0RzRbR{((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyt__init__s          cC s`t|dddt}|j\}}|jjd}||ks\td||fn|S(NRURVRis:Incorrect number of features. Got %d features, expected %d(R RRtcluster_centers_RD(RR-R2R3texpected_n_features((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyt_check_test_datas cC st|j}t|d|jd|d|jd|jd|jd|jd|jd|j d |d |j d |j d |j d t  \|_|_|_|_|S(s|Compute k-means clustering. Parameters ---------- X : array-like or sparse matrix, shape=(n_samples, n_features) Training instances to cluster. It must be noted that the data will be converted to C ordering, which will cause a memory copy if the given data is not C-contiguous. y : Ignored not used, present here for API consistency by convention. sample_weight : array-like, shape (n_samples,), optional The weights for each observation in X. If None, all observations are assigned equal weight (default: None) R.RPR^RyR]R_R`RJR0RzRbR{R|(RR0RR.R^RyR]R_R`RJRzRbR{R'Rtlabels_tinertia_R(RR-tyRPR0((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pytfits !cC s|j|d|jS(sCompute cluster centers and predict cluster index for each sample. Convenience method; equivalent to calling fit(X) followed by predict(X). Parameters ---------- X : {array-like, sparse matrix}, shape = [n_samples, n_features] New data to transform. y : Ignored not used, present here for API consistency by convention. sample_weight : array-like, shape (n_samples,), optional The weights for each observation in X. If None, all observations are assigned equal weight (default: None) Returns ------- labels : array, shape [n_samples,] Index of the cluster each sample belongs to. RP(RR(RR-RRP((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyt fit_predictscC s|j|d|j|S(sCompute clustering and transform X to cluster-distance space. Equivalent to fit(X).transform(X), but more efficiently implemented. Parameters ---------- X : {array-like, sparse matrix}, shape = [n_samples, n_features] New data to transform. y : Ignored not used, present here for API consistency by convention. sample_weight : array-like, shape (n_samples,), optional The weights for each observation in X. If None, all observations are assigned equal weight (default: None) Returns ------- X_new : array, shape [n_samples, k] X transformed in the new space. RP(Rt _transform(RR-RRP((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyt fit_transformscC s)t|d|j|}|j|S(sTransform X to a cluster-distance space. In the new space, each dimension is the distance to the cluster centers. Note that even if X is sparse, the array returned by `transform` will typically be dense. Parameters ---------- X : {array-like, sparse matrix}, shape = [n_samples, n_features] New data to transform. Returns ------- X_new : array, shape [n_samples, k] X transformed in the new space. R(RRR(RR-((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyt transforms cC st||jS(s-guts of transform method; no input validation(RR(RR-((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyRscC sHt|d|j|}t|dt}t||||jdS(sPredict the closest cluster each sample in X belongs to. In the vector quantization literature, `cluster_centers_` is called the code book and each value returned by `predict` is the index of the closest code in the code book. Parameters ---------- X : {array-like, sparse matrix}, shape = [n_samples, n_features] New data to predict. sample_weight : array-like, shape (n_samples,), optional The weights for each observation in X. If None, all observations are assigned equal weight (default: None) Returns ------- labels : array, shape [n_samples,] Index of the cluster each sample belongs to. RRi(RRRR'RR(RR-RPR/((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pytpredicts   cC sIt|d|j|}t|dt}t||||jd S(s[Opposite of the value of X on the K-means objective. Parameters ---------- X : {array-like, sparse matrix}, shape = [n_samples, n_features] New data. y : Ignored not used, present here for API consistency by convention. sample_weight : array-like, shape (n_samples,), optional The weights for each observation in X. If None, all observations are assigned equal weight (default: None) Returns ------- score : float Opposite of the value of X on the K-means objective. RRi(RRRR'RR(RR-RRPR/((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pytscore9s   N(t__name__t __module__t__doc__RR'RRRRRRRRR(((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyRs       g{Gz?c  C st||||d|\} } |ro| dkrot| } || |jk}|jd|jdkrtj|td|jd}t||`. random_reassign : boolean, optional If True, centers with very low counts are randomly reassigned to observations. reassignment_ratio : float, optional Control the fraction of the maximum number of counts for a center to be reassigned. A higher value means that low count centers are more likely to be reassigned, which means that the model will take longer to converge, but should converge in a better clustering. verbose : bool, optional, default False Controls the verbosity. compute_squared_diff : bool If set to False, the squared diff computation is skipped. old_center_buffer : int Copy of old centers for monitoring convergence. Returns ------- inertia : float Sum of squared distances of samples to their closest cluster center. squared_diff : numpy array, shape (n_clusters,) Squared distances between previous and updated cluster centers. Rig?treplaceRas1[MiniBatchKMeans] Reassigning %i cluster centers.gRG(RRRtR(RRtargsortR tFalsetchoiceR#R$R ROtintptwheretminRt_mini_batch_update_csrR)R&traveltdot(R-RPR/R4t weight_sumstold_center_buffertcompute_squared_diffRtrandom_reassignR0treassignment_ratioR_tnearest_centerRt to_reassigntindices_dont_reassignt n_reassignst new_centersRt squared_difft center_idxt center_masktwsumtdiff((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyt_mini_batch_stepUsXC   $            ic C s||j}||j}|jd} |jd} | d krS|} |} n_t|jd|d} | dkrdn| } | d| || } | d| || } |rd|d||| f} | GHn|dkr| |kr|rd|d|fGHntS|jd } |jd d }| d ksK| | krZd }| } n |d7}|jd k r||jkr|rd |d|fGHntS| |d<| |d<| |d <||d `. Parameters ---------- n_clusters : int, optional, default: 8 The number of clusters to form as well as the number of centroids to generate. init : {'k-means++', 'random' or an ndarray}, default: 'k-means++' Method for initialization, defaults to 'k-means++': 'k-means++' : selects initial cluster centers for k-mean clustering in a smart way to speed up convergence. See section Notes in k_init for more details. 'random': choose k observations (rows) at random from data for the initial centroids. If an ndarray is passed, it should be of shape (n_clusters, n_features) and gives the initial centers. max_iter : int, optional Maximum number of iterations over the complete dataset before stopping independently of any early stopping criterion heuristics. batch_size : int, optional, default: 100 Size of the mini batches. verbose : boolean, optional Verbosity mode. compute_labels : boolean, default=True Compute label assignment and inertia for the complete dataset once the minibatch optimization has converged in fit. random_state : int, RandomState instance or None (default) Determines random number generation for centroid initialization and random reassignment. Use an int to make the randomness deterministic. See :term:`Glossary `. tol : float, default: 0.0 Control early stopping based on the relative center changes as measured by a smoothed, variance-normalized of the mean center squared position changes. This early stopping heuristics is closer to the one used for the batch variant of the algorithms but induces a slight computational and memory overhead over the inertia heuristic. To disable convergence detection based on normalized center change, set tol to 0.0 (default). max_no_improvement : int, default: 10 Control early stopping based on the consecutive number of mini batches that does not yield an improvement on the smoothed inertia. To disable convergence detection based on inertia, set max_no_improvement to None. init_size : int, optional, default: 3 * batch_size Number of samples to randomly sample for speeding up the initialization (sometimes at the expense of accuracy): the only algorithm is initialized by running a batch KMeans on a random subset of the data. This needs to be larger than n_clusters. n_init : int, default=3 Number of random initializations that are tried. In contrast to KMeans, the algorithm is only run once, using the best of the ``n_init`` initializations as measured by inertia. reassignment_ratio : float, default: 0.01 Control the fraction of the maximum number of counts for a center to be reassigned. A higher value means that low count centers are more easily reassigned, which means that the model will take longer to converge, but should converge in a better clustering. Attributes ---------- cluster_centers_ : array, [n_clusters, n_features] Coordinates of cluster centers labels_ : Labels of each point (if compute_labels is set to True). inertia_ : float The value of the inertia criterion associated with the chosen partition (if compute_labels is set to True). The inertia is defined as the sum of square distances of samples to their nearest neighbor. Examples -------- >>> from sklearn.cluster import MiniBatchKMeans >>> import numpy as np >>> X = np.array([[1, 2], [1, 4], [1, 0], ... [4, 2], [4, 0], [4, 4], ... [4, 5], [0, 1], [2, 2], ... [3, 2], [5, 5], [1, -1]]) >>> # manually fit on batches >>> kmeans = MiniBatchKMeans(n_clusters=2, ... random_state=0, ... batch_size=6) >>> kmeans = kmeans.partial_fit(X[0:6,:]) >>> kmeans = kmeans.partial_fit(X[6:12,:]) >>> kmeans.cluster_centers_ array([[1, 1], [3, 4]]) >>> kmeans.predict([[0, 0], [4, 4]]) array([0, 1], dtype=int32) >>> # fit on the whole data >>> kmeans = MiniBatchKMeans(n_clusters=2, ... random_state=0, ... batch_size=6, ... max_iter=10).fit(X) >>> kmeans.cluster_centers_ array([[3.95918367, 2.40816327], [1.12195122, 1.3902439 ]]) >>> kmeans.predict([[0, 0], [4, 4]]) array([1, 0], dtype=int32) See also -------- KMeans The classic implementation of the clustering method based on the Lloyd's algorithm. It consumes the whole set of input data at each iteration. Notes ----- See http://www.eecs.tufts.edu/~dsculley/papers/fastkmeans.pdf is k-means++idigi ig{Gz?c C sntt|jd|d|d|d|d|d|d| | |_||_||_| |_| |_dS(NR.R^R]R_R0RJRy(tsuperRRRRtcompute_labelsRR( RR.R^R]RR_RR0RJRRRyR((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyRs    cC sot|j}t|dddddtjtjg}|j\}}||jkrstd||jfnt ||}|j }t |j drtj |j d|j|_ |dkrtjd |j td d d}qnt|d t}|jd kr>t||j} tj|d|j} nd } tjdd|j} tj|jd|j} ttjt||j} t|j| } |j}|dkrd|j}n||kr|}n||_|jd||}||}||}||}d}x%t |D]}|j!r`d|d||j fGHntj|jd|j}t"||j|j d|d|d|}t#||||||| t$ddd|j!\}}t%||||\}}|j!rd|d||fGHn|dks1||kr5||_&||_'|}q5q5Wi}xt | D]}|jd||j}t#|||||||j&|j'| | d kd| d|ddt|j'j(dkd|d|j)d|j!\}}t*||| | ||||d|j!rcPqcqcW|d|_+|j,rk|j-||\|_.|_/n|S(sCompute the centroids on X by chunking it into mini-batches. Parameters ---------- X : array-like or sparse matrix, shape=(n_samples, n_features) Training instances to cluster. It must be noted that the data will be converted to C ordering, which will cause a memory copy if the given data is not C-contiguous. y : Ignored not used, present here for API consistency by convention. sample_weight : array-like, shape (n_samples,), optional The weights for each observation in X. If None, all observations are assigned equal weight (default: None) RURVRWRTRs'n_samples=%d should be >= n_clusters=%dRYisiExplicit initial center position passed: performing only one init in MiniBatchKMeans instead of n_init=%dRZiRgiisInit %d/%d with method: %sR0R/RRR_sInertia for init %d/%d: %fRi RN(0RR0R RRfRgRR.RDRRRyRjR^RRRlRmRnRR'RJRLRRR tceilRR]RRt init_size_R"R)R_RRRRRtcounts_RRRRRt_labels_inertia_minibatchRR(RR-RRPR0R2R3RyR/RJRRt n_batchesRRtvalidation_indicestX_validtsample_weight_validtx_squared_norms_validRtinit_idxRtcluster_centersRRt_Rtconvergence_contextRtminibatch_indices((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyRs   "                       !c C s|jrdGHnt||}t|dt}t|jd|j}g|D]*}t|||||||j^qR}t |\}}t j |t j |fS(sZCompute labels and inertia using mini batches. This is slightly slower than doing everything at once but preventes memory errors / segfaults. Parameters ---------- X : array-like, shape (n_samples, n_features) Input data. sample_weight : array-like, shape (n_samples,) The weights for each observation in X. Returns ------- labels : array, shape (n_samples,) Cluster labels for each point. inertia : float Sum of squared distances of points to nearest cluster. s,Computing label assignment and total inertiaRi( R_RRRR'R RRRRRuRthstackR(( RR-RPR/tslicestsRRR((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyRWs 4c C st|dddd}|j\}}t|jdrZtj|jd|j|_n|dkrj|St||}t|dt }t |d t |j |_ t|d  st|d  r)t||j|jd |j d |d|j|_tj|jd|j|_t}d}nH|j jdd|jjdk}tj|jdd|j}t||||j|jtjdd|jdd|d|d |j d|jd|j|jrt||||j\|_|_n|S(s?Update k means estimate on a single mini-batch X. Parameters ---------- X : array-like, shape = [n_samples, n_features] Coordinates of the data points to cluster. It must be noted that X will be copied if it is not C-contiguous. y : Ignored not used, present here for API consistency by convention. sample_weight : array-like, shape (n_samples,), optional The weights for each observation in X. If None, all observations are assigned equal weight (default: None) RURVRWRTRYRiRt random_state_RRR0R/Ri iRRRR_N(R RRjR^RRRRRRR'tgetattrRR0RRR.RRRRRRR"RRRR_RRRR( RR-RRPR2R3R/RR((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyt partial_fitwsD!             $cC s0t|d|j|}|j||dS(sPredict the closest cluster each sample in X belongs to. In the vector quantization literature, `cluster_centers_` is called the code book and each value returned by `predict` is the index of the closest code in the code book. Parameters ---------- X : {array-like, sparse matrix}, shape = [n_samples, n_features] New data to predict. sample_weight : array-like, shape (n_samples,), optional The weights for each observation in X. If None, all observations are assigned equal weight (default: None) Returns ------- labels : array, shape [n_samples,] Index of the cluster each sample belongs to. Ri(RRR(RR-RP((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyRs N( RRRR'RRRRRR(((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pyR&s     B(=Rt __future__RRltnumpyRt scipy.sparsetsparseR#tbaseRRRtmetrics.pairwiseRRt utils.extmathRRRtutils.sparsefuncs_fastR tutils.sparsefuncsR tutils.validationR tutilsR R RRRt utils._joblibRRRt externals.sixRt exceptionsRtRt_k_means_elkanRRRBRFRLRRRR'RRpRoRRRRRRR(((s7lib/python2.7/site-packages/sklearn/cluster/k_means_.pytsd    h       ~ 1; OU @