\c @sdZddlmZmZmZmZddlZddlZddl m Z ddl m Z ddl mZmZddlmZdd lmZmZdd lmZdd lmZd d lmZd dlmZddlmZddlm Z dZ!dZ"dde$dZ%ddddde$dZ&dZ'dZ(dZ)e*de%de'de(de)Z+dZ,deefd YZ-d!e-efd"YZ.dS(#sHierarchical Agglomerative Clustering These routines perform some hierarchical agglomerative clustering of some input data. Authors : Vincent Michel, Bertrand Thirion, Alexandre Gramfort, Gael Varoquaux License: BSD 3 clause i(theapifytheappoptheappusht heappushpopN(tsparse(tconnected_componentsi(t BaseEstimatort ClusterMixin(tsix(tpaired_distancestpairwise_distances(t check_array(t check_memoryi(t _hierarchical(tAgglomerationTransform(t IntFloatDict(txrangecCs|jd}|jd|ks3|jd|krRtd|j|jfn||j}tj|stj|stj|}q|j}nt|\}}|dkrt j d|ddxt |D]}t j ||kd}||}xt |D]} t j || kd} || } t|| d|} t j | t j| k\} }| d} |d}t||| | |f 1. Completing it to avoid stopping the tree early.t stacklevelitmetric(tshapet ValueErrortTRtisspmatrix_lilt isspmatrixt lil_matrixttolilRtwarningstwarnRtnptwhereR tmintTrue(tXt connectivitytaffinityt n_samplest n_componentstlabelstitidx_itXitjtidx_jtXjtDtiitjj((s;lib/python2.7/site-packages/sklearn/cluster/hierarchical.pyt_fix_connectivity!s8      $  #cCsddlm}|jd}tjd|jjj}||j|jdk<||j}|j }d|j|j|k`. Parameters ---------- X : array, shape (n_samples, n_features) feature matrix representing n_samples samples to be clustered connectivity : sparse matrix (optional). connectivity matrix. Defines for each sample the neighboring samples following a given structure of the data. The matrix is assumed to be symmetric and only the upper triangular half is used. Default is None, i.e, the Ward algorithm is unstructured. n_clusters : int (optional) Stop early the construction of the tree at n_clusters. This is useful to decrease computation time if the number of clusters is not small compared to the number of samples. In this case, the complete tree is not computed, thus the 'children' output is of limited use, and the 'parents' output should rather be used. This option is valid only when specifying a connectivity matrix. return_distance : bool (optional) If True, return the distance between the clusters. Returns ------- children : 2D array, shape (n_nodes-1, 2) The children of each non-leaf node. Values less than `n_samples` correspond to leaves of the tree which are the original samples. A node `i` greater than or equal to `n_samples` is a non-leaf node and has children `children_[i - n_samples]`. Alternatively at the i-th iteration, children[i][0] and children[i][1] are merged to form node `n_samples + i` n_components : int The number of connected components in the graph. n_leaves : int The number of leaves in the tree parents : 1D array, shape (n_nodes, ) or None The parent of each node. Only returned when a connectivity matrix is specified, elsewhere 'None' is returned. distances : 1D array, shape (n_nodes-1, ) Only returned if return_distance is set to True (for compatibility). The distances between the centers of the nodes. `distances[i]` corresponds to a weighted euclidean distance between the nodes `children[i, 1]` and `children[i, 2]`. If the nodes refer to leaves of the tree, then `distances[i]` is their unweighted euclidean distance. Distances are updated in the following way (from scipy.hierarchy.linkage): The new entry :math:`d(u,v)` is computed as follows, .. math:: d(u,v) = \sqrt{\frac{|v|+|s|} {T}d(v,s)^2 + \frac{|v|+|t|} {T}d(v,t)^2 - \frac{|v|} {T}d(s,t)^2} where :math:`u` is the newly joined cluster consisting of clusters :math:`s` and :math:`t`, :math:`v` is an unused cluster in the forest, :math:`T=|v|+|s|+|t|`, and :math:`|*|` is the cardinality of its argument. This is also known as the incremental algorithm. ii(t hierarchysPartial build of the tree is implemented only for structured clustering (i.e. with explicit connectivity). The algorithm will build the full tree and only retain the lower branches required for the specified number of clustersRit requirementstWNR"t euclideans]Cannot provide more clusters than samples. %i n_clusters was asked, and there are %i samples.R2tordertCig@(ii(,RtasarraytndimtreshapeRRCt scipy.clusterRQRRtrequiretwardR4RAR/RRBtrowstappendtextendtlentarraytzerostemptyR1R tcompute_ward_disttlistRtmovestzipRR@tonestbooltint8trangeRRtFalsetfillt _get_parentsRtsqrt("R R!RERFR#t n_featuresRQtoutRKROR$RDt coord_rowt coord_coltAtindR;R&t moments_1t moments_2tinertiaRLt used_nodetchildrent not_visitedtktinertR)tlt n_additionstinitidxtn_leavestc((s;lib/python2.7/site-packages/sklearn/cluster/hierarchical.pyt ward_treesQ    (    %  $    $ !    8&t deprecatedtcompleteRTc"s(|dkrtjdtntj|}|jdkrRtj|d}n|j\}}itj d6tj d6dd6} y| |} Wn-t k rt d| j|fnX|dkr8dd lm} |dk rtjd d d n|d kr?tj|jddd\} } || | f}nw|dkrTd}nb|dkrid}nMt|r||}tj|jddd\} } || | f}n| j|d|d|}|dddd fjtj}|r(|ddd f}|d|d|fS|d|dfSt||d|\}}|j}|j|jk}|j||_|j||_|j||_~|d kr||j|jfjd}n#t||j||jd|}||_|dkrd |d}n ||ks-td ||}|dkr`t||||||S|r|tj||}ntj|dt}t }|j!}xt"t#|j|j$D]o\\}}t%tj|dtj&tj|dtj'|<|j(fdt#||DqW~t)|tj*|dtj&}tj+|dtj&}g}xVt,||D]E}x4t-rt.|}||j/r||j0rPqqW|j/} |j0} |r|j1|||`. Parameters ---------- X : array, shape (n_samples, n_features) feature matrix representing n_samples samples to be clustered connectivity : sparse matrix (optional). connectivity matrix. Defines for each sample the neighboring samples following a given structure of the data. The matrix is assumed to be symmetric and only the upper triangular half is used. Default is None, i.e, the Ward algorithm is unstructured. n_components : int (optional) The number of connected components in the graph. n_clusters : int (optional) Stop early the construction of the tree at n_clusters. This is useful to decrease computation time if the number of clusters is not small compared to the number of samples. In this case, the complete tree is not computed, thus the 'children' output is of limited use, and the 'parents' output should rather be used. This option is valid only when specifying a connectivity matrix. linkage : {"average", "complete", "single"}, optional, default: "complete" Which linkage criteria to use. The linkage criterion determines which distance to use between sets of observation. - average uses the average of the distances of each observation of the two sets - complete or maximum linkage uses the maximum distances between all observations of the two sets. - single uses the minimum of the distances between all observations of the two sets. affinity : string or callable, optional, default: "euclidean". which metric to use. Can be "euclidean", "manhattan", or any distance know to paired distance (see metric.pairwise) return_distance : bool, default False whether or not to return the distances between the clusters. Returns ------- children : 2D array, shape (n_nodes-1, 2) The children of each non-leaf node. Values less than `n_samples` correspond to leaves of the tree which are the original samples. A node `i` greater than or equal to `n_samples` is a non-leaf node and has children `children_[i - n_samples]`. Alternatively at the i-th iteration, children[i][0] and children[i][1] are merged to form node `n_samples + i` n_components : int The number of connected components in the graph. n_leaves : int The number of leaves in the tree. parents : 1D array, shape (n_nodes, ) or None The parent of each node. Only returned when a connectivity matrix is specified, elsewhere 'None' is returned. distances : ndarray, shape (n_nodes-1,) Returned when return_distance is set to True. distances[i] refers to the distance between children[i][0] and children[i][1] when they are merged. See also -------- ward_tree : hierarchical clustering with ward linkage Rs:n_components was deprecated in 0.19will be removed in 0.21iiRtaveragetsinglesEUnknown linkage option, linkage should be one of %s, but %s was given(RQsPartial build of the tree is implemented only for structured clustering (i.e. with explicit connectivity). The algorithm will build the full tree and only retain the lower branches required for the specified number of clustersRit precomputediR|tl2RTtl1t manhattant cityblocktmethodRNR"R1R2c3s6|],\}}|krtj||VqdS(N(R t WeightedEdge(t.0trtd(Ru(s;lib/python2.7/site-packages/sklearn/cluster/hierarchical.pys s(ii(RR(7RRtDeprecationWarningRRWRXRYRR t max_merget average_mergeRCtKeyErrorRtkeysRZRQt triu_indicestcallabletlinkageR4R?R/R9R;R<R6R tAssertionErrorRPRctobjectReRRBRgR]RRAR1R_RR@RhRRRtatbtweightR^RlRRRa("R R!R$RERR"RFR#Rptlinkage_choicest join_funcRQR&R)RqRKROt diag_maskRDRtRxR6R;RLRyRzR|tedgetn_itn_jRsR~RR((Rus;lib/python2.7/site-packages/sklearn/cluster/hierarchical.pyt linkage_treeWsQ            "      "(   "                    (cOsd|d`. Parameters ---------- n_clusters : int, default=2 The number of clusters to find. affinity : string or callable, default: "euclidean" Metric used to compute the linkage. Can be "euclidean", "l1", "l2", "manhattan", "cosine", or 'precomputed'. If linkage is "ward", only "euclidean" is accepted. memory : None, str or object with the joblib.Memory interface, optional Used to cache the output of the computation of the tree. By default, no caching is done. If a string is given, it is the path to the caching directory. connectivity : array-like or callable, optional Connectivity matrix. Defines for each sample the neighboring samples following a given structure of the data. This can be a connectivity matrix itself or a callable that transforms the data into a connectivity matrix, such as derived from kneighbors_graph. Default is None, i.e, the hierarchical clustering algorithm is unstructured. compute_full_tree : bool or 'auto' (optional) Stop early the construction of the tree at n_clusters. This is useful to decrease computation time if the number of clusters is not small compared to the number of samples. This option is useful only when specifying a connectivity matrix. Note also that when varying the number of clusters and using caching, it may be advantageous to compute the full tree. linkage : {"ward", "complete", "average", "single"}, optional (default="ward") Which linkage criterion to use. The linkage criterion determines which distance to use between sets of observation. The algorithm will merge the pairs of cluster that minimize this criterion. - ward minimizes the variance of the clusters being merged. - average uses the average of the distances of each observation of the two sets. - complete or maximum linkage uses the maximum distances between all observations of the two sets. - single uses the minimum of the distances between all observations of the two sets. pooling_func : callable, default='deprecated' Ignored. .. deprecated:: 0.20 ``pooling_func`` has been deprecated in 0.20 and will be removed in 0.22. Attributes ---------- labels_ : array [n_samples] cluster labels for each point n_leaves_ : int Number of leaves in the hierarchical tree. n_components_ : int The estimated number of connected components in the graph. children_ : array-like, shape (n_samples-1, 2) The children of each non-leaf node. Values less than `n_samples` correspond to leaves of the tree which are the original samples. A node `i` greater than or equal to `n_samples` is a non-leaf node and has children `children_[i - n_samples]`. Alternatively at the i-th iteration, children[i][0] and children[i][1] are merged to form node `n_samples + i` Examples -------- >>> from sklearn.cluster import AgglomerativeClustering >>> import numpy as np >>> X = np.array([[1, 2], [1, 4], [1, 0], ... [4, 2], [4, 4], [4, 0]]) >>> clustering = AgglomerativeClustering().fit(X) >>> clustering # doctest: +NORMALIZE_WHITESPACE AgglomerativeClustering(affinity='euclidean', compute_full_tree='auto', connectivity=None, linkage='ward', memory=None, n_clusters=2, pooling_func='deprecated') >>> clustering.labels_ array([1, 1, 1, 0, 0, 0]) iRTtautoR\RcCsC||_||_||_||_||_||_||_dS(N(REtmemoryR!tcompute_full_treeRR"t pooling_func(tselfRER"RR!RRR((s;lib/python2.7/site-packages/sklearn/cluster/hierarchical.pyt__init__s      c Cs|jdkr2t|t r2tjdtnt|ddd|}t|j}|j dkrt dt |j n|j dkr|j d krt d |j fn|j tkrt d |j tjfnt|j }|j}|jdk rPt|jr2|j|}nt|d d ddg}nt|}|j}|jdkr}t}n|dkr|j tdd|k}n|j }|rd}ni} |j dkr|j | d<|j | d`. Parameters ---------- n_clusters : int, default 2 The number of clusters to find. affinity : string or callable, default "euclidean" Metric used to compute the linkage. Can be "euclidean", "l1", "l2", "manhattan", "cosine", or 'precomputed'. If linkage is "ward", only "euclidean" is accepted. memory : None, str or object with the joblib.Memory interface, optional Used to cache the output of the computation of the tree. By default, no caching is done. If a string is given, it is the path to the caching directory. connectivity : array-like or callable, optional Connectivity matrix. Defines for each feature the neighboring features following a given structure of the data. This can be a connectivity matrix itself or a callable that transforms the data into a connectivity matrix, such as derived from kneighbors_graph. Default is None, i.e, the hierarchical clustering algorithm is unstructured. compute_full_tree : bool or 'auto', optional, default "auto" Stop early the construction of the tree at n_clusters. This is useful to decrease computation time if the number of clusters is not small compared to the number of features. This option is useful only when specifying a connectivity matrix. Note also that when varying the number of clusters and using caching, it may be advantageous to compute the full tree. linkage : {"ward", "complete", "average", "single"}, optional (default="ward") Which linkage criterion to use. The linkage criterion determines which distance to use between sets of features. The algorithm will merge the pairs of cluster that minimize this criterion. - ward minimizes the variance of the clusters being merged. - average uses the average of the distances of each feature of the two sets. - complete or maximum linkage uses the maximum distances between all features of the two sets. - single uses the minimum of the distances between all observations of the two sets. pooling_func : callable, default np.mean This combines the values of agglomerated features into a single value, and should accept an array of shape [M, N] and the keyword argument `axis=1`, and reduce it to an array of size [M]. Attributes ---------- labels_ : array-like, (n_features,) cluster labels for each feature. n_leaves_ : int Number of leaves in the hierarchical tree. n_components_ : int The estimated number of connected components in the graph. children_ : array-like, shape (n_nodes-1, 2) The children of each non-leaf node. Values less than `n_features` correspond to leaves of the tree which are the original samples. A node `i` greater than or equal to `n_features` is a non-leaf node and has children `children_[i - n_features]`. Alternatively at the i-th iteration, children[i][0] and children[i][1] are merged to form node `n_features + i` Examples -------- >>> import numpy as np >>> from sklearn import datasets, cluster >>> digits = datasets.load_digits() >>> images = digits.images >>> X = np.reshape(images, (len(images), -1)) >>> agglo = cluster.FeatureAgglomeration(n_clusters=32) >>> agglo.fit(X) # doctest: +ELLIPSIS FeatureAgglomeration(affinity='euclidean', compute_full_tree='auto', connectivity=None, linkage='ward', memory=None, n_clusters=32, pooling_func=...) >>> X_reduced = agglo.transform(X) >>> X_reduced.shape (1797, 32) iRTRR\c CsDtt|jd|d|d|d|d|d|||_dS(NRERR!RRR"(tsuperRRR(RRER"RR!RRR((s;lib/python2.7/site-packages/sklearn/cluster/hierarchical.pyRs  cKs=t|ddddgddd|}tj||j|S(sFit the hierarchical clustering on the data Parameters ---------- X : array-like, shape = [n_samples, n_features] The data y : Ignored Returns ------- self RRtcscRtensure_min_featuresiR(R RRR(RR Rtparams((s;lib/python2.7/site-packages/sklearn/cluster/hierarchical.pyRscCs tdS(N(tAttributeError(R((s;lib/python2.7/site-packages/sklearn/cluster/hierarchical.pyt fit_predictsN( RRRRCRtmeanRRtpropertyR(((s;lib/python2.7/site-packages/sklearn/cluster/hierarchical.pyRMs\ (/RtheapqRRRRRtnumpyRtscipyRR3RtbaseRRt externalsRtmetrics.pairwiseR R tutilsR tutils.validationR tR t_feature_agglomerationRtutils.fast_dictRtexternals.six.movesRR/RPRCRlRRRRRtdictRRRR(((s;lib/python2.7/site-packages/sklearn/cluster/hierarchical.pyt s>"   2 5      1