ó ù`]c@ súddlmZmZmZddlmZddlZddlmZddl m Z yddl Z Wne k r{nXddl mZddlmZdefd „ƒYZd „Zd „Zd efd „ƒYZedefd„ƒYƒZdS(iÿÿÿÿ(tprint_functiontunicode_literalstdivision(tabstractmethodN(tstdout(tsqrt(tClusterI(tpython_2_unicode_compatibletVectorSpaceClusterercB sweZdZed d„Zeed„Zed„ƒZd„Z ed„ƒZ d„Z d„Z d„Z d „ZRS( u© Abstract clusterer which takes tokens and maps them into a vector space. Optionally performs singular value decomposition to reduce the dimensionality. cC sd|_||_||_dS(u) :param normalise: should vectors be normalised to length 1 :type normalise: boolean :param svd_dimensions: number of dimensions to use in reducing vector dimensionsionality with SVD :type svd_dimensions: int N(tNonet_Ttt_should_normaliset_svd_dimensions(tselft normalisetsvd_dimensions((s0lib/python2.7/site-packages/nltk/cluster/util.pyt__init__s  c C sVt|ƒdkst‚|jr<tt|j|ƒƒ}n|jr|jt|dƒkrtjj tj tj |ƒƒƒ\}}}||j tj |jtj ƒ}|dd…d|j…f}|d|j…dd…f} tj tj|| ƒƒ}tj |ƒ|_n|j||ƒ|rRg|D]} |j| ƒ^q9SdS(Ni(tlentAssertionErrorR tlisttmapt _normaliseR tnumpytlinalgtsvdt transposetarraytidentitytfloat64tdotR tcluster_vectorspacetclassify( R tvectorstassign_clustersttracetutdtvttStTtDttvector((s0lib/python2.7/site-packages/nltk/cluster/util.pytcluster+s "-cC sdS(uD Finds the clusters using the given set of vectors. N((R R R"((s0lib/python2.7/site-packages/nltk/cluster/util.pyRDtcC s^|jr|j|ƒ}n|jdk rBtj|j|ƒ}n|j|ƒ}|j|ƒS(N(R RR R RRtclassify_vectorspacet cluster_name(R R)R*((s0lib/python2.7/site-packages/nltk/cluster/util.pyRJs  cC sdS(uN Returns the index of the appropriate cluster for the vector. N((R R)((s0lib/python2.7/site-packages/nltk/cluster/util.pyR,RR+cC sR|jr|j|ƒ}n|jdk rBtj|j|ƒ}n|j||ƒS(N(R RR R RRtlikelihood_vectorspace(R R)tlabel((s0lib/python2.7/site-packages/nltk/cluster/util.pyt likelihoodXs  cC s#|j|ƒ}||krdSdS(uP Returns the likelihood of the vector belonging to the cluster. gð?g(R,(R R)R*t predicted((s0lib/python2.7/site-packages/nltk/cluster/util.pyR._scC sF|jr|j|ƒ}n|jdk rBtj|j|ƒ}n|S(uU Returns the vector after normalisation and dimensionality reduction N(R RR R RR(R R)((s0lib/python2.7/site-packages/nltk/cluster/util.pyR)fs  cC s|ttj||ƒƒS(u7 Normalises the vector to unit length. (RRR(R R)((s0lib/python2.7/site-packages/nltk/cluster/util.pyRpsN(t__name__t __module__t__doc__tFalseR RR*RRRR,R0R.R)R(((s0lib/python2.7/site-packages/nltk/cluster/util.pyRs     cC s ||}ttj||ƒƒS(u} Returns the euclidean distance between vectors u and v. This is equivalent to the length of the vector (u - v). (RRR(R#tvtdiff((s0lib/python2.7/site-packages/nltk/cluster/util.pyteuclidean_distancews cC s@dtj||ƒttj||ƒƒttj||ƒƒS(us Returns 1 minus the cosine of the angle between vectors v and u. This is equal to 1 - (u.v / |u||v|). i(RRR(R#R6((s0lib/python2.7/site-packages/nltk/cluster/util.pytcosine_distance€st_DendrogramNodecB s5eZdZd„Zed„Zd„Zd„ZRS(u Tree node of a dendrogram. cG s||_||_dS(N(t_valuet _children(R tvaluetchildren((s0lib/python2.7/site-packages/nltk/cluster/util.pyR‹s cC sX|jr=g}x'|jD]}|j|j|ƒƒqW|S|rM|jgS|gSdS(N(R<textendtleavesR;(R tvaluesR@tchild((s0lib/python2.7/site-packages/nltk/cluster/util.pyR@s  cC sä|j|fg}x›t|ƒ|kr¯|jƒ\}}|jsY|j||fƒPnxF|jD];}|jr‹|j|j|fƒqc|jd|fƒqcW|jƒqWg}x'|D]\}}|j|jƒƒq½W|S(Ni(R;RtpopR<tpushtappendtsortR@(R tntqueuetprioritytnodeRBtgroups((s0lib/python2.7/site-packages/nltk/cluster/util.pyRKšs  cC st|j|jƒdkS(Ni(R9R;(R t comparator((s0lib/python2.7/site-packages/nltk/cluster/util.pyt__lt__¯s(R2R3R4RtTrueR@RKRM(((s0lib/python2.7/site-packages/nltk/cluster/util.pyR:ˆs   t DendrogramcB sAeZdZgd„Zd„Zd„Zgd„Zd„ZRS(u Represents a dendrogram, a tree with a specified branching order. This must be initialised with the leaf items, then iteratively call merge for each branch. This class constructs a tree representing the order of calls to the merge function. cC sDg|D]}t|ƒ^q|_tj|jƒ|_d|_dS(us :param items: the items at the leaves of the dendrogram :type items: sequence of (any) iN(R:t_itemstcopyt_original_itemst_merge(R titemstitem((s0lib/python2.7/site-packages/nltk/cluster/util.pyR¼s"cG s‡t|ƒdkst‚t|jg|D]}|j|^q(Œ}|jd7_||j|d R+u uc3 s|]}|jˆƒVqdS(N(tcenter(t.0RU(twidth(s0lib/python2.7/site-packages/nltk/cluster/util.pys &sN(u+u-u|(RRPR:RSRRR;tmaxRtintRCRR<tindextmintrangeRERFtjoin(R t leaf_labelstJOINtHLINKtVLINKRYR@tlast_rowtleafR_RbRHt verticalsRIRJtchild_left_leafRVtmin_idxtmax_idxRWRBtvertical((R]R^Rgs0lib/python2.7/site-packages/nltk/cluster/util.pytshowásV               #cC sWt|jƒdkr-t|j|jŒ}n |jd}|jtƒ}dt|ƒS(Niiu(RRPR:RSR@R5(R RYR@((s0lib/python2.7/site-packages/nltk/cluster/util.pyt__repr__)s  (R2R3R4RRXRKRyRz(((s0lib/python2.7/site-packages/nltk/cluster/util.pyRO³s   H(t __future__RRRtabcRRQtsysRtmathRRt ImportErrortnltk.cluster.apiRt nltk.compatRRR8R9tobjectR:RO(((s0lib/python2.7/site-packages/nltk/cluster/util.pyts   _ +