ó ù`]c@sddlmZmZddlZddlZddlZddlZddlZddlm Z ddl m Z ddl m Z mZmZddlmZddlmZddlmZd „Zd „Zd „Zd efd „ƒYZedkrddlZejƒndS(iÿÿÿÿ(tprint_functiontunicode_literalsN(t text_type(tZipFilePathPointer(tfind_dirt find_filetfind_jars_within_path(tParserI(tDependencyGraph(ttaggedsents_to_conllcCswddlm}|d1d2d3d4d5d6d7d8d9d:d;d<d=d>d?d@dAdBdCdDdEdFdGdHdIdJdKdLgƒ}|jS(MNiÿÿÿÿ(t RegexpTaggeru\.$u.u\,$u,u\?$u?u\($u(u\)$u)u\[$u[u\]$u]u^-?[0-9]+(.[0-9]+)?$uCDu(The|the|A|a|An|an)$uDTu&(He|he|She|she|It|it|I|me|Me|You|you)$uPRPu(His|his|Her|her|Its|its)$uPRP$u(my|Your|your|Yours|yours)$u (on|On|in|In|at|At|since|Since)$uINu (for|For|ago|Ago|before|Before)$u(till|Till|until|Until)$u(by|By|beside|Beside)$u(under|Under|below|Below)$u(over|Over|above|Above)$u (across|Across|through|Through)$u(into|Into|towards|Towards)$u(onto|Onto|from|From)$u.*able$uJJu.*ness$uNNu.*ly$uRBu.*s$uNNSu.*ing$uVBGu.*ed$uVBDu.*(u\.$u.(u\,$u,(u\?$u?(u\($u((u\)$u)(u\[$u[(u\]$u](u^-?[0-9]+(.[0-9]+)?$uCD(u(The|the|A|a|An|an)$uDT(u&(He|he|She|she|It|it|I|me|Me|You|you)$uPRP(u(His|his|Her|her|Its|its)$uPRP$(u(my|Your|your|Yours|yours)$uPRP$(u (on|On|in|In|at|At|since|Since)$uIN(u (for|For|ago|Ago|before|Before)$uIN(u(till|Till|until|Until)$uIN(u(by|By|beside|Beside)$uIN(u(under|Under|below|Below)$uIN(u(over|Over|above|Above)$uIN(u (across|Across|through|Through)$uIN(u(into|Into|towards|Towards)$uIN(u(onto|Onto|from|From)$uIN(u.*able$uJJ(u.*ness$uNN(u.*ly$uRB(u.*s$uNNS(u.*ing$uVBG(u.*ed$uVBD(u.*uNN(tnltk.tagR ttag(R t_tagger((s.lib/python2.7/site-packages/nltk/parse/malt.pytmalt_regex_taggers> cCs¶tjj|ƒr|}nt|dd ƒ}dddg}tt|ƒƒ}td„|Dƒƒ}tdddgƒ}|j|ƒsŽt‚tt d„|ƒƒs¬t‚t |ƒS( uE A module to find MaltParser .jar file and its dependencies. tenv_varsu MALT_PARSERucss%|]}tjj|ƒdVqdS(iN(tostpathtsplit(t.0tjar((s.lib/python2.7/site-packages/nltk/parse/malt.pys Nsu log4j.jaru libsvm.jaruliblinear-1.8.jarcSs|jdƒo|jdƒS(Nu maltparser-u.jar(t startswithtendswith(ti((s.lib/python2.7/site-packages/nltk/parse/malt.pytSt(u MALT_PARSER( RRtexistsRtsetRtissubsettAssertionErrortanytfiltertlist(tparser_dirnamet _malt_dirtmalt_dependenciest _malt_jarst_jars((s.lib/python2.7/site-packages/nltk/parse/malt.pytfind_maltparserCs cCs@|dkrdStjj|ƒr&|St|dddtƒSdS(u8 A module to find pre-trained MaltParser model. u malt_temp.mcoRu MALT_MODELtverboseN(u MALT_MODEL(tNoneRRRRtFalse(tmodel_filename((s.lib/python2.7/site-packages/nltk/parse/malt.pytfind_malt_modelXs  t MaltParsercBsweZdZd d d d„Zedd„Zedd„Zd d d„Ze ed„ƒZ ed„Z ed„Z RS( uØ A class for dependency parsing with MaltParser. The input is the paths to: - a maltparser directory - (optionally) the path to a pre-trained MaltParser .mco model file - (optionally) the tagger to use for POS tagging before parsing - (optionally) additional Java arguments Example: >>> from nltk.parse import malt >>> # With MALT_PARSER and MALT_MODEL environment set. >>> mp = malt.MaltParser('maltparser-1.7.2', 'engmalt.linear-1.7.mco') # doctest: +SKIP >>> mp.parse_one('I shot an elephant in my pajamas .'.split()).tree() # doctest: +SKIP (shot I (elephant an) (in (pajamas my)) .) >>> # Without MALT_PARSER and MALT_MODEL environment. >>> mp = malt.MaltParser('/home/user/maltparser-1.7.2/', '/home/user/engmalt.linear-1.7.mco') # doctest: +SKIP >>> mp.parse_one('I shot an elephant in my pajamas .'.split()).tree() # doctest: +SKIP (shot I (elephant an) (in (pajamas my)) .) cCs|t|ƒ|_|dk r!|ng|_t|ƒ|_|jdk|_tjƒ|_ |dk rl|nt ƒ|_ dS(u¿ An interface for parsing with the Malt Parser. :param parser_dirname: The path to the maltparser directory that contains the maltparser-1.x.jar :type parser_dirname: str :param model_filename: The name of the pre-trained model with .mco file extension. If provided, training will not be required. (see http://www.maltparser.org/mco/mco.html and see http://www.patful.com/chalk/node/185) :type model_filename: str :param tagger: The tagger used to POS tag the raw string before formatting to CONLL format. It should behave like `nltk.pos_tag` :type tagger: function :param additional_java_args: This is the additional Java arguments that one can use when calling Maltparser, usually this is the heapsize limits, e.g. `additional_java_args=['-Xmx1024m']` (see http://goo.gl/mpDBvQ) :type additional_java_args: list u malt_temp.mcoN( R&t malt_jarsR(tadditional_java_argsR+tmodelt_trainedttempfilet gettempdirt working_dirRttagger(tselfR!R*R4R.((s.lib/python2.7/site-packages/nltk/parse/malt.pyt__init__xs unullc csÇ|jstdƒ‚ntjddd|jdddtƒb}tjddd|jdddtƒ2}x't|ƒD]}|jt|ƒƒqyW|j ƒ|j |j |j dd ƒ}t j ƒ}y$t jt jj|jƒd ƒWnnX|j||ƒ} t j|ƒ| d k rEtd d j|ƒ| fƒ‚nt|j ƒA} x7| jƒjd ƒD] } tt| d|ƒgƒVqmWWdQXWdQXWdQXt j|j ƒt j|j ƒdS(u· Use MaltParser to parse multiple POS tagged sentences. Takes multiple sentences where each sentence is a list of (word, tag) tuples. The sentences must have already been tokenized and tagged. :param sentences: Input sentences to parse :type sentence: list(list(tuple(str, str))) :return: iter(iter(``DependencyGraph``)) the dependency graph representation of each sentence u0Parser has not been trained. Call train() first.tprefixumalt_input.conll.tdirtmodeuwtdeleteumalt_output.conll.uparseiu0MaltParser parsing (%s) failed with exit code %du u ttop_relation_labelN(R0t ExceptionR1tNamedTemporaryFileR3R)R twriteRtclosetgenerate_malt_commandtnameRtgetcwdtchdirRRR/t_executetjointopentreadtiterRtremove( R5t sentencesR'R;t input_filet output_filetlinetcmdt _current_pathtrettinfilettree_str((s.lib/python2.7/site-packages/nltk/parse/malt.pytparse_tagged_sents¢sD       $  *cs,‡fd†|Dƒ}ˆj||d|ƒS(un Use MaltParser to parse multiple sentences. Takes a list of sentences, where each sentence is a list of words. Each sentence will be automatically tagged with this MaltParser instance's tagger. :param sentences: Input sentences to parse :type sentence: list(list(str)) :return: iter(DependencyGraph) c3s|]}ˆj|ƒVqdS(N(R4(Rtsentence(R5(s.lib/python2.7/site-packages/nltk/parse/malt.pys ïsR;(RS(R5RJR'R;ttagged_sentences((R5s.lib/python2.7/site-packages/nltk/parse/malt.pyt parse_sentsäs cCsîdg}||j7}tjjdƒr.dnd}|d|j|jƒg7}|dg7}tjj|j ƒr˜|dtjj |j ƒdg7}n|d|j g7}|d |g7}|d krÚ|d |g7}n|d |g7}|S( u This function generates the maltparser command use at the terminal. :param inputfilename: path to the input file :type inputfilename: str :param outputfilename: path to the output file :type outputfilename: str ujavauwinu;u:u-cpuorg.maltparser.Maltu-ciÿÿÿÿu-iuparseu-ou-m( R.tsystplatformRRER-RRRR/R(R5t inputfilenametoutputfilenameR9RNtclasspaths_separator((s.lib/python2.7/site-packages/nltk/parse/malt.pyR@ôs   & cCs:|r dntj}tj|d|d|ƒ}|jƒS(Ntstdouttstderr(R(t subprocesstPIPEtPopentwait(RNR'toutputtp((s.lib/python2.7/site-packages/nltk/parse/malt.pyRDsc Cs†tjddd|jdddtƒ3}djd„|Dƒƒ}|jt|ƒƒWd QX|j|jd |ƒt j |jƒd S( uÍ Train MaltParser from a list of ``DependencyGraph`` objects :param depgraphs: list of ``DependencyGraph`` objects for training input data :type depgraphs: DependencyGraph R7umalt_train.conll.R8R9uwR:u css|]}|jdƒVqdS(i N(tto_conll(Rtdg((s.lib/python2.7/site-packages/nltk/parse/malt.pys &sNR'( R1R=R3R)RER>Rttrain_from_fileRARRI(R5t depgraphsR'RKt input_str((s.lib/python2.7/site-packages/nltk/parse/malt.pyttrains c Csît|tƒrŒtjddd|jdddtƒQ}|jƒ&}|jƒ}|jt |ƒƒWdQX|j |j d|ƒSWdQXn|j |dd ƒ}|j ||ƒ}|d krátd d j|ƒ|fƒ‚nt|_dS( u— Train MaltParser from a file :param conll_file: str for the filename of the training input data :type conll_file: str R7umalt_train.conll.R8R9uwR:NR'ulearniu1MaltParser training (%s) failed with exit code %du (t isinstanceRR1R=R3R)RFRGR>RRfRAR@RDR<REtTrueR0(R5t conll_fileR'RKtconll_input_filet conll_strRNRP((s.lib/python2.7/site-packages/nltk/parse/malt.pyRf-s    N( t__name__t __module__t__doc__R(R6R)RSRVR@t staticmethodRDRiRf(((s.lib/python2.7/site-packages/nltk/parse/malt.pyR,ds %B  u__main__(t __future__RRRRWR1R^tinspecttsixRt nltk.dataRtnltk.internalsRRRtnltk.parse.apiRtnltk.parse.dependencygraphRtnltk.parse.utilR RR&R+R,Rotdoctestttestmod(((s.lib/python2.7/site-packages/nltk/parse/malt.pyt s$      &  æ C