B >?ð[¯&ã@sÂddlmZmZmZddlmZddlZddlmZddl m Z y ddl Z Wne k r`YnXddl mZddlmZGdd „d eƒZd d „Zd d „ZGdd„deƒZeGdd„deƒƒZdS)é)Úprint_functionÚunicode_literalsÚdivision)ÚabstractmethodN)Ústdout)Úsqrt)ÚClusterI)Úpython_2_unicode_compatiblec@sdeZdZdZddd„Zddd„Zedd „ƒZd d „Zed d „ƒZ dd„Z dd„Z dd„Z dd„Z dS)ÚVectorSpaceClustererz© Abstract clusterer which takes tokens and maps them into a vector space. Optionally performs singular value decomposition to reduce the dimensionality. FNcCsd|_||_||_dS)a) :param normalise: should vectors be normalised to length 1 :type normalise: boolean :param svd_dimensions: number of dimensions to use in reducing vector dimensionsionality with SVD :type svd_dimensions: int N)Ú_TtÚ_should_normaliseÚ_svd_dimensions)ÚselfZ normaliseZsvd_dimensions©rú0lib/python3.7/site-packages/nltk/cluster/util.pyÚ__init__szVectorSpaceClusterer.__init__c sêt|ƒdkst‚ˆjr&ttˆj|ƒƒ}ˆjrĈjt|dƒkrÄtj  t  t  |¡¡¡\}}}|dˆj…t  ˆjtj ¡}|dd…dˆj…f}|dˆj…dd…f} t  t || ¡¡}t  |¡ˆ_ˆ ||¡|ræ‡fdd„|DƒSdS)Nrcsg|]}ˆ |¡‘qSr)Úclassify)Ú.0Úvector)rrrú Bsz0VectorSpaceClusterer.cluster..)ÚlenÚAssertionErrorr ÚlistÚmapÚ _normaliser ÚnumpyZlinalgZsvdZ transposeZarrayZidentityZfloat64Údotr Úcluster_vectorspace) rÚvectorsZassign_clustersÚtraceÚuÚdZvtÚSÚTZDtr)rrÚcluster+s  zVectorSpaceClusterer.clustercCsdS)zD Finds the clusters using the given set of vectors. Nr)rrrrrrrDsz(VectorSpaceClusterer.cluster_vectorspacecCs<|jr| |¡}|jdk r(t |j|¡}| |¡}| |¡S)N)r rr rrÚclassify_vectorspaceZ cluster_name)rrr$rrrrJs    zVectorSpaceClusterer.classifycCsdS)zN Returns the index of the appropriate cluster for the vector. Nr)rrrrrr%Rsz)VectorSpaceClusterer.classify_vectorspacecCs4|jr| |¡}|jdk r(t |j|¡}| ||¡S)N)r rr rrÚlikelihood_vectorspace)rrZlabelrrrÚ likelihoodXs   zVectorSpaceClusterer.likelihoodcCs| |¡}||krdSdS)zP Returns the likelihood of the vector belonging to the cluster. gð?g)r%)rrr$Z predictedrrrr&_s z+VectorSpaceClusterer.likelihood_vectorspacecCs,|jr| |¡}|jdk r(t |j|¡}|S)zU Returns the vector after normalisation and dimensionality reduction N)r rr rr)rrrrrrfs   zVectorSpaceClusterer.vectorcCs|tt ||¡ƒS)z7 Normalises the vector to unit length. )rrr)rrrrrrpszVectorSpaceClusterer._normalise)FN)FF)Ú__name__Ú __module__Ú __qualname__Ú__doc__rr$rrrr%r'r&rrrrrrr s    r cCs||}tt ||¡ƒS)z} Returns the euclidean distance between vectors u and v. This is equivalent to the length of the vector (u - v). )rrr)r ÚvZdiffrrrÚeuclidean_distancewsr-cCs0dt ||¡tt ||¡ƒtt ||¡ƒS)zs Returns 1 minus the cosine of the angle between vectors v and u. This is equal to 1 - (u.v / |u||v|). é)rrr)r r,rrrÚcosine_distance€sr/c@s2eZdZdZdd„Zd dd„Zdd„Zd d „Zd S) Ú_DendrogramNodez Tree node of a dendrogram. cGs||_||_dS)N)Ú_valueÚ _children)rÚvalueZchildrenrrrr‹sz_DendrogramNode.__init__TcCsD|jr.g}x|jD]}| | |¡¡qW|S|r:|jgS|gSdS)N)r2ÚextendÚleavesr1)rÚvaluesr5Úchildrrrr5s z_DendrogramNode.leavescCs¦|j|fg}xpt|ƒ|kr|| ¡\}}|js<| ||f¡Px4|jD]*}|jr`| |j|f¡qD| d|f¡qDW| ¡qWg}x|D]\}}| | ¡¡qˆW|S)Nr)r1rÚpopr2ÚpushÚappendÚsortr5)rÚnÚqueueÚpriorityÚnoder7Úgroupsrrrr@šs    z_DendrogramNode.groupscCst|j|jƒdkS)Nr)r/r1)rZ comparatorrrrÚ__lt__¯sz_DendrogramNode.__lt__N)T)r(r)r*r+rr5r@rArrrrr0ˆs  r0c@s@eZdZdZgfdd„Zdd„Zdd„Zgfdd „Zd d „Zd S) Ú Dendrograma Represents a dendrogram, a tree with a specified branching order. This must be initialised with the leaf items, then iteratively call merge for each branch. This class constructs a tree representing the order of calls to the merge function. cCs(dd„|Dƒ|_t |j¡|_d|_dS)zs :param items: the items at the leaves of the dendrogram :type items: sequence of (any) cSsg|] }t|ƒ‘qSr)r0)rÚitemrrrrÁsz'Dendrogram.__init__..r.N)Ú_itemsÚcopyÚ_original_itemsÚ_merge)rÚitemsrrrr¼szDendrogram.__init__cslt|ƒdkst‚tˆjf‡fdd„|DƒžŽ}ˆjd7_|ˆj|d<x|dd…D] }ˆj|=qXWdS)a= Merges nodes at given indices in the dendrogram. The nodes will be combined which then replaces the first node specified. All other nodes involved in the merge will be removed. :param indices: indices of the items to merge (at least two) :type indices: seq of int écsg|]}ˆj|‘qSr)rD)rÚi)rrrrÏsz$Dendrogram.merge..r.rN)rrr0rGrD)rÚindicesr?rJr)rrÚmergeÅs zDendrogram.mergecCs6t|jƒdkr"t|jf|jžŽ}n |jd}| |¡S)z’ Finds the n-groups of items (leaves) reachable from a cut at depth n. :param n: number of groups :type n: int r.r)rrDr0rGr@)rr<Úrootrrrr@Õs zDendrogram.groupscs@d\}}}t|jƒdkr,t|jf|jžŽ}n |jd}|j}|rF|}ndd„|Dƒ}ttt|ƒƒd‰ˆd‰tˆˆdƒ‰d‡‡fdd „ ‰d d „}|j|fg} ‡fd d„|Dƒ} x`| r|   ¡\} } t td d„| j ƒƒ} t t|j | ƒƒ}| rt |ƒ}t|ƒ}x´tt|ƒƒD]¤}||| krz||kr@|ˆ|d|ƒƒn,||kr\|ˆ||dƒƒn|ˆ|||ƒƒˆ|ƒ| |<n:||kr’|kr¨nn|ˆ|||ƒƒn || |ƒqW|dƒx(| j D]}|j rÊ|  |j|f¡qÊW|  ¡x| D]}||ƒqúW|dƒq¸W|d ‡fdd„|Dƒ¡ƒ|dƒdS)z× Print the dendrogram in ASCII art to standard out. :param leaf_labels: an optional list of strings to use for labeling the leaves :type leaf_labels: list )ú+ú-ú|r.rcSsg|]}d|j‘qS)z%s)r1)rÚleafrrrrösz#Dendrogram.show..rIú csdˆ|||ˆfS)Nz%s%s%sr)ZcentreÚleftÚright)ÚlhalfÚrhalfrrÚformatþszDendrogram.show..formatcSst |¡dS)N)rÚwrite)ÚstrrrrÚdisplaysz Dendrogram.show..displaycsg|] }ˆdƒ‘qS)rRr)rrQ)rWrrrscSs| d¡dS)NFr)r5)ÚcrrrÚ sz!Dendrogram.show..Ú Úc3s|]}| ˆ¡VqdS)N)Úcenter)rrC)Úwidthrrú &sz"Dendrogram.show..N)rRrR)rrDr0rGrFÚmaxrÚintr1r8rr2ÚindexÚminÚranger:r;Újoin)rZ leaf_labelsZJOINZHLINKZVLINKrMr5Zlast_rowrZr=Z verticalsr>r?Zchild_left_leafrKZmin_idxZmax_idxrJr7Zverticalr)rWrUrVr`rÚshowásV          zDendrogram.showcCsBt|jƒdkr"t|jf|jžŽ}n |jd}| d¡}dt|ƒS)Nr.rFz)rrDr0rGr5)rrMr5rrrÚ__repr__)s   zDendrogram.__repr__N) r(r)r*r+rrLr@rhrirrrrrB³s   HrB)Z __future__rrrÚabcrrEÚsysrZmathrrÚ ImportErrorZnltk.cluster.apirZ nltk.compatr r r-r/Úobjectr0rBrrrrÚs       _ +