B P?[?@sddlmZmZddlZddlZddlZddlZddlZddlm Z ddl m Z ddl m Z mZmZddlmZddlmZddlmZd d Zd d Zd dZGdddeZedkrddlZedS))print_functionunicode_literalsN) text_type)ZipFilePathPointer)find_dir find_filefind_jars_within_path)ParserI)DependencyGraph)taggedsents_to_conllcCsRddlm}|ddddddd d d d d dddddddddddddddddg}|jS)Nr) RegexpTagger)z\.$.)z\,$,)z\?$?)z\($()z\)$))z\[$[)z\]$])z^-?[0-9]+(.[0-9]+)?$ZCD)z(The|the|A|a|An|an)$ZDT)z&(He|he|She|she|It|it|I|me|Me|You|you)$ZPRP)z(His|his|Her|her|Its|its)$zPRP$)z(my|Your|your|Yours|yours)$zPRP$)z (on|On|in|In|at|At|since|Since)$IN)z (for|For|ago|Ago|before|Before)$r)z(till|Till|until|Until)$r)z(by|By|beside|Beside)$r)z(under|Under|below|Below)$r)z(over|Over|above|Above)$r)z (across|Across|through|Through)$r)z(into|Into|towards|Towards)$r)z(onto|Onto|from|From)$r)z.*able$ZJJ)z.*ness$NN)z.*ly$ZRB)z.*s$ZNNS)z.*ing$ZVBG)z.*ed$ZVBD)z.*r)Znltk.tagr tag)r Z_taggerr.lib/python3.7/site-packages/nltk/parse/malt.pymalt_regex_taggers> rcCstj|r|}n t|dd}dddg}tt|}tdd|D}tdddg}||sbttt d d |sxtt |S) zE A module to find MaltParser .jar file and its dependencies. )Z MALT_PARSER)env_varscss|]}tj|dVqdS)N)ospathsplit).0Zjarrrr Nsz"find_maltparser..z log4j.jarz libsvm.jarzliblinear-1.8.jarcSs|do|dS)Nz maltparser-z.jar) startswithendswith)irrrSsz!find_maltparser..) rrexistsrsetrissubsetAssertionErroranyfilterlist)parser_dirnameZ _malt_dirZmalt_dependenciesZ _malt_jarsZ_jarsrrrfind_maltparserCs    r.cCs.|dkr dStj|r|St|dddSdS)z8 A module to find pre-trained MaltParser model. Nz malt_temp.mco)Z MALT_MODELF)rverbose)rrr&r)model_filenamerrrfind_malt_modelXs  r1c@sZeZdZdZdddZdddZdd d Zdd d Zedd dZ dddZ dddZ dS) MaltParsera A class for dependency parsing with MaltParser. The input is the paths to: - a maltparser directory - (optionally) the path to a pre-trained MaltParser .mco model file - (optionally) the tagger to use for POS tagging before parsing - (optionally) additional Java arguments Example: >>> from nltk.parse import malt >>> # With MALT_PARSER and MALT_MODEL environment set. >>> mp = malt.MaltParser('maltparser-1.7.2', 'engmalt.linear-1.7.mco') # doctest: +SKIP >>> mp.parse_one('I shot an elephant in my pajamas .'.split()).tree() # doctest: +SKIP (shot I (elephant an) (in (pajamas my)) .) >>> # Without MALT_PARSER and MALT_MODEL environment. >>> mp = malt.MaltParser('/home/user/maltparser-1.7.2/', '/home/user/engmalt.linear-1.7.mco') # doctest: +SKIP >>> mp.parse_one('I shot an elephant in my pajamas .'.split()).tree() # doctest: +SKIP (shot I (elephant an) (in (pajamas my)) .) NcCsTt||_|dk r|ng|_t||_|jdk|_t|_|dk rH|nt |_ dS)a An interface for parsing with the Malt Parser. :param parser_dirname: The path to the maltparser directory that contains the maltparser-1.x.jar :type parser_dirname: str :param model_filename: The name of the pre-trained model with .mco file extension. If provided, training will not be required. (see http://www.maltparser.org/mco/mco.html and see http://www.patful.com/chalk/node/185) :type model_filename: str :param tagger: The tagger used to POS tag the raw string before formatting to CONLL format. It should behave like `nltk.pos_tag` :type tagger: function :param additional_java_args: This is the additional Java arguments that one can use when calling Maltparser, usually this is the heapsize limits, e.g. `additional_java_args=['-Xmx1024m']` (see http://goo.gl/mpDBvQ) :type additional_java_args: list Nz malt_temp.mco) r. malt_jarsadditional_java_argsr1model_trainedtempfileZ gettempdir working_dirrtagger)selfr-r0r9r4rrr__init__xs     zMaltParser.__init__Fnullc csP|jstdtjd|jddd }tjd|jddd}xt|D]}|t|qFW||j |j |j dd}t }yt t j|jd Wn YnX|||} t || d k rtd d || ft|j 2} x*| d D]} tt| |d gVqWWdQRXWdQRXWdQRXt |j t |j dS)a Use MaltParser to parse multiple POS tagged sentences. Takes multiple sentences where each sentence is a list of (word, tag) tuples. The sentences must have already been tokenized and tagged. :param sentences: Input sentences to parse :type sentence: list(list(tuple(str, str))) :return: iter(iter(``DependencyGraph``)) the dependency graph representation of each sentence z0Parser has not been trained. Call train() first.zmalt_input.conll.wF)prefixdirmodedeletezmalt_output.conll.parse)r@rz0MaltParser parsing (%s) failed with exit code %d z )top_relation_labelN)r6 Exceptionr7NamedTemporaryFiler8r writerclosegenerate_malt_commandnamergetcwdchdirrrr5_executejoinopenreaditerr remove) r: sentencesr/rD input_fileZ output_filelinecmdZ _current_pathretZinfileZtree_strrrrparse_tagged_sentssD    2 zMaltParser.parse_tagged_sentscs"fdd|D}j|||dS)an Use MaltParser to parse multiple sentences. Takes a list of sentences, where each sentence is a list of words. Each sentence will be automatically tagged with this MaltParser instance's tagger. :param sentences: Input sentences to parse :type sentence: list(list(str)) :return: iter(DependencyGraph) c3s|]}|VqdS)N)r9)r Zsentence)r:rrr!sz)MaltParser.parse_sents..)rD)rX)r:rSr/rDZtagged_sentencesr)r:r parse_sentss zMaltParser.parse_sentscCsdg}||j7}tjdr dnd}|d||jg7}|dg7}tj|j rl|dtj |j dg7}n|d|j g7}|d |g7}|d kr|d |g7}|d |g7}|S) a This function generates the maltparser command use at the terminal. :param inputfilename: path to the input file :type inputfilename: str :param outputfilename: path to the output file :type outputfilename: str javawin;:z-cpzorg.maltparser.Maltz-cz-irBz-oz-m) r4sysplatformr"rNr3rrr&r5r)r:Z inputfilenameZoutputfilenamer@rVZclasspaths_separatorrrrrIs      z MaltParser.generate_malt_commandcCs&|rdntj}tj|||d}|S)N)stdoutstderr) subprocessPIPEPopenwait)rVr/outputprrrrMszMaltParser._executec Csbtjd|jddd(}ddd|D}|t|WdQRX|j|j|d t |jdS) z Train MaltParser from a list of ``DependencyGraph`` objects :param depgraphs: list of ``DependencyGraph`` objects for training input data :type depgraphs: DependencyGraph zmalt_train.conll.r=F)r>r?r@rA css|]}|dVqdS) N)Zto_conll)r Zdgrrrr!&sz#MaltParser.train..N)r/) r7rFr8rNrGrtrain_from_filerJrrR)r:Z depgraphsr/rTZ input_strrrrtrains zMaltParser.trainc Cst|tr`tjd|jddd<}|}|}|t|WdQRX|j |j |dSQRX|j |dd}| ||}|d krt d d ||fd |_dS) z Train MaltParser from a file :param conll_file: str for the filename of the training input data :type conll_file: str zmalt_train.conll.r=F)r>r?r@rAN)r/Zlearn)r@rz1MaltParser training (%s) failed with exit code %drCT) isinstancerr7rFr8rOrPrGrrkrJrIrMrErNr6)r:Z conll_filer/rTZconll_input_fileZ conll_strrVrWrrrrk-s   zMaltParser.train_from_file)NNN)Fr<)Fr<)NN)F)F)F) __name__ __module__ __qualname____doc__r;rXrYrI staticmethodrMrlrkrrrrr2ds % B    r2__main__)Z __future__rrrr_r7rcinspectZsixrZ nltk.datarZnltk.internalsrrrZnltk.parse.apir Znltk.parse.dependencygraphr Znltk.parse.utilr rr.r1r2rnZdoctestZtestmodrrrr s&     & gC