B >?[- @s*ddlmZmZddlZddlZddlmZmZddlZddl Z ddl m Z m Z ej ddkZerddZddlZejZejZddlmZejZdd lmZnd dZydd lmZWn ek rdd lmZYnXeZdd lmZmZedZed dZGdddeZeZddlZddl Z ddlZGdddZ!ddl"Z#ddlZ$ddlm%Z%Gddde&ZddddgZ'dde'DZ(ddZ)ddZ*d d!Z+ydd"l,m,Z-WnBek rydd"l.m,Z-Wnek re+Z-YnXYnXd#d$Z/d%d&Z0d'd(Z1d)d*Z2d+d,Z3Gd-d.d.ej4Z4dS)/)absolute_importprint_functionN)update_wrapperwraps) string_types text_typecCs|jjS)N)__self__ __class__)methr *lib/python3.7/site-packages/nltk/compat.py get_im_classsr)timezone)TemporaryDirectorycCs|jS)N)Zim_class)r r r r r(s)StringIO)tzinfo timedelta)Zhoursc@s(eZdZdZddZddZddZdS)UTCcCstS)N)ZERO)selfdtr r r utcoffset:sz UTC.utcoffsetcCsdS)Nrr )rrr r r tzname=sz UTC.tznamecCstS)N)r)rrr r r dst@szUTC.dstN)__name__ __module__ __qualname____doc__rrrr r r r r7src@s2eZdZdZejddfddZddZdd Zd S) UnicodeWriterz A CSV writer which will write rows to CSV file "f", which is encoded in the given encoding. see https://docs.python.org/2/library/csv.html zutf-8replacecKsDt|_tj|jfd|i||_||_t|}||d|_dS)Ndialect)errors) cStringIOrqueuecsvwriterstreamcodecsgetincrementalencoderencoder)rfr"encodingr#kwdsZ encoder_clsr r r __init__Ps   zUnicodeWriter.__init__cCst|tr|dS|SdS)Nzutf-8) isinstancerencode)rdatar r r r1Zs  zUnicodeWriter.encodecsXjfdd|Dj}|d}j|d}j|j ddS)Ncsg|]}|qSr )r1).0s)rr r asz*UnicodeWriter.writerow..zutf-8r!r) r'writerowr%getvaluedecoder+r1r(writetruncate)rrowr2r )rr r6`s    zUnicodeWriter.writerowN) rrrrr&Zexcelr/r1r6r r r r r Is r )mkdtempc@seZdZdZdddZddZd d Zdd d ZddZddZ e e j Z e e jjZe e jjZe e jjZe e jZe e jZejZddZdS)raCreate and return a temporary directory. This has the same behavior as mkdtemp but can be used as a context manager. For example: with TemporaryDirectory() as tmpdir: ... Upon exiting the context, the directory and everything contained in it are removed. http://stackoverflow.com/questions/19296146/tempfile-temporarydirectory-context-manager-in-python-2-7 tmpNcCsd|_d|_t||||_dS)NF)_closednamer<)rsuffixprefixdirr r r r/~szTemporaryDirectory.__init__cCsd|jj|jS)Nz <{} {!r}>)formatr rr@)rr r r __repr__szTemporaryDirectory.__repr__cCs|jS)N)r@)rr r r __enter__szTemporaryDirectory.__enter__Fc Cs|jr|jsy||jWnJttfk rf}z(dt|kr@td||tj ddSd}~XYnXd|_|r| d|t dS)NNonez"ERROR: {!r} while cleaning up {!r})fileTzImplicitly cleaning up {!r}) r@r?_rmtree TypeErrorAttributeErrorstrprintrDsysstderr_warnWarning)rrPexr r r cleanups    zTemporaryDirectory.cleanupcCs |dS)N)rS)rexcvaluetbr r r __exit__szTemporaryDirectory.__exit__cCs|jdddS)NT)rP)rS)rr r r __del__szTemporaryDirectory.__del__c Csx||D]x}|||}y||o2|| }Wntk rNd}YnX|r`||q y||Wq tk rYq Xq Wy||Wntk rYnXdS)NF)_listdir _path_join_isdir_islinkOSErrorrI_remove_rmdir)rpathr@fullnameisdirr r r rIs     zTemporaryDirectory._rmtree)r=r>N)F)rrrrr/rErFrSrWrX staticmethod_oslistdirrYr`joinrZrbr[islinkr\remover^rmdirr_ _warningswarnrPrIr r r r rps         r)ZchunkersZmaxent_ne_chunker)helpZtagsets)ZtaggersZmaxent_treebank_pos_tagger)Z tokenizersZpunktcCsg|]}tjj|qSr )osr`rf)r3 path_listr r r r5sr5cCsztrvxptD]h}|t|kr dt|kr ||t|}|||ddkrT|d7}|d|d||d}Pq W|S)Nz/PY3z.zip)PY3_PY3_DATA_UPDATESrLindexlen)r`itemposr r r add_py3_datas rvcsfdd}t|S)Ncs*|dt|df|dd}||S)Nrr)rv)argskwargs) init_funcr r _decorators zpy3_data.._decorator)r)rzr{r )rzr py3_datas r|cs:t|tr|d}tjdfddtd|DS)Nasciir=c3s|]}|dkr|VqdS)ZMnNr )r3c)categoryr r sz!remove_accents..ZNFKD)r0bytesr8 unicodedatarrfZ normalize)textr )rr remove_accentss   r) unidecodecCs^t|tstdt|js8|j|_ts8tt|j|_t|j sZ|j |_ tsZt|j |_ |S)a This decorator defines __unicode__ method and fixes __repr__ and __str__ methods under Python 2. To support Python 2 and 3 with a single code base, define __str__ and __repr__ methods returning unicode text and apply this decorator to the class. Original __repr__ and __str__ would be available as unicode_repr and __unicode__ (under both Python 2 and Python 3). z1This decorator doesn't work for old-style classes) issubclassobject ValueError _was_fixed__str__Z __unicode__rp_7bit_transliteratedrE unicode_repr)klassr r r python_2_unicode_compatibles    rcCs@tr t|St|dr|St|tr8t|ddSt|S)a0 For classes that was fixed with @python_2_unicode_compatible ``unicode_repr`` returns ``obj.unicode_repr()``; for unicode strings the result is returned without "u" letter (to make output the same under Python 2.x and Python 3.x); for other variables it is the same as ``repr``. rrN)rpreprhasattrrr0r)objr r r r*s  rcs8fdd}t|ddgtdr.j|_d|_|S)Ncs t|S)N) transliterate)r)methodr r wrapper@sz _transliterated..wrapperrr_nltk_compat_7bitT)rrr_nltk_compat_transliterated)rrr )rr r?s   rcs8fdd}t|ddgtdr.j|_d|_|S)Ncs|ddS)Nr}backslashreplace)r1)r)rr r rLsz_7bit..wrapperrrrT)rrrr)rrr )rr rKs   rcCst|ddpt|ddS)NrFr)getattr)rr r r rXsrcs"eZdZdZdfdd ZZS)Fractiona This is a simplified backwards compatible version of fractions.Fraction from Python >=3.5. It adds the `_normalize` parameter such that it does not normalize the denominator to the Greatest Common Divisor (gcd) when the numerator is 0. This is most probably only used by the nltk.translate.bleu_score.py where numerator and denominator of the different ngram precisions are mutable. But the idea of "mutable" fraction might not be applicable to other usages, See http://stackoverflow.com/questions/34561265 This objects should be deprecated once NLTK stops supporting Python < 3.5 See https://github.com/nltk/nltk/issues/1330 rNTcs8tt||||}|s4t|tkr4|r4||_||_|S)N)superr__new__typeintZ _numeratorZ _denominator)cls numerator denominatorZ _normalize)r r r rns zFraction.__new__)rNT)rrrrr __classcell__r r )r r r^sr)5Z __future__rrrmrN functoolsrrZ fractionsrZsixrr version_inforpriorBytesIOZdatetimerZutcrZtempfilerr$ ImportErrorrrrZHOURr&r)r warningsrjrdr<rZ DATA_UPDATESrqrvr|rrrZtext_unidecoderrrrrrr r r r  sn   # Z  %