B P?ð[¼*ã @s²ddlmZddlmZddlmZdgZy$ddlmZddl m Z dd„Z Wn.e k r|Z ze fd d„Z Wd d Z [ XYnXd d „Zd d„ZGdd„deƒZdd„Zdd„Zd S)é)Úprint_function)ÚParserI)ÚTreeÚ BllipParser)ÚRerankingParser)Úget_unified_model_parameterscCsdS)N©rrrú/lib/python3.7/site-packages/nltk/parse/bllip.pyÚ_ensure_bllip_import_or_error\sr cCstd|ƒ‚dS)Nz&Couldn't import bllipparser module: %s)Ú ImportError)Úierrr r bsNcCsNy$xt|ƒD]\}}| d¡q WWn$tk rHtd||fƒ‚YnXdS)NÚasciizTToken %d (%r) is non-ASCII. BLLIP Parser currently doesn't support non-ASCII inputs.)Ú enumerateÚdecodeÚUnicodeDecodeErrorÚ ValueError)ÚwordsÚiÚwordrrr Ú _ensure_asciifsrcCst t|jƒ¡S)N)rZ fromstringÚstrZ ptb_parse)Ú scored_parserrr Ú_scored_parse_to_nltk_treeqsrc@s8eZdZdZd dd„Zdd„Zdd„Zed d d „ƒZdS) rzÑ Interface for parsing with BLLIP Parser. BllipParser objects can be constructed with the ``BllipParser.from_unified_model_dir`` class method or manually using the ``BllipParser`` constructor. NcCsRtƒ|p i}|pi}tƒ|_|jj|f|Ž|rN|rN|jjf||dœ|—ŽdS)a´ Load a BLLIP Parser model from scratch. You'll typically want to use the ``from_unified_model_dir()`` class method to construct this object. :param parser_model: Path to parser model directory :type parser_model: str :param reranker_features: Path the reranker model's features file :type reranker_features: str :param reranker_weights: Path the reranker model's weights file :type reranker_weights: str :param parser_options: optional dictionary of parser options, see ``bllipparser.RerankingParser.RerankingParser.load_parser_options()`` for more information. :type parser_options: dict(str) :param reranker_options: optional dictionary of reranker options, see ``bllipparser.RerankingParser.RerankingParser.load_reranker_model()`` for more information. :type reranker_options: dict(str) )Zfeatures_filenameZweights_filenameN)r rÚrrpZload_parser_modelZload_reranker_model)ÚselfZ parser_modelZreranker_featuresZreranker_weightsÚparser_optionsÚreranker_optionsrrr Ú__init__|s!zBllipParser.__init__ccs0t|ƒ|j |¡}x|D]}t|ƒVqWdS)a› Use BLLIP Parser to parse a sentence. Takes a sentence as a list of words; it will be automatically tagged with this BLLIP Parser instance's tagger. :return: An iterator that generates parse trees for the sentence from most likely to least likely. :param sentence: The sentence to be parsed :type sentence: list(str) :rtype: iter(Tree) N)rrÚparser)rÚsentenceÚ nbest_listrrrr r«s   zBllipParser.parsec csng}i}x2t|ƒD]&\}\}}| |¡|dk r|||<qWt|ƒ|j ||¡}x|D]}t|ƒVqXWdS)aœ Use BLLIP to parse a sentence. Takes a sentence as a list of (word, tag) tuples; the sentence must have already been tokenized and tagged. BLLIP will attempt to use the tags provided but may use others if it can't come up with a complete parse subject to those constraints. You may also specify a tag as ``None`` to leave a token's tag unconstrained. :return: An iterator that generates parse trees for the sentence from most likely to least likely. :param sentence: Input sentence to parse as (word, tag) pairs :type sentence: list(tuple(str, str)) :rtype: iter(Tree) N)rÚappendrrZ parse_taggedr) rZword_and_tag_pairsrZtag_maprrÚtagr rrrr Ú tagged_parse½s   zBllipParser.tagged_parsecCst|ƒ\}}}||||||ƒS)a Create a ``BllipParser`` object from a unified parsing model directory. Unified parsing model directories are a standardized way of storing BLLIP parser and reranker models together on disk. See ``bllipparser.RerankingParser.get_unified_model_parameters()`` for more information about unified model directories. :return: A ``BllipParser`` object using the parser and reranker models in the model directory. :param model_dir: Path to the unified model directory. :type model_dir: str :param parser_options: optional dictionary of parser options, see ``bllipparser.RerankingParser.RerankingParser.load_parser_options()`` for more information. :type parser_options: dict(str) :param reranker_options: optional dictionary of reranker options, see ``bllipparser.RerankingParser.RerankingParser.load_reranker_model()`` for more information. :type reranker_options: dict(str) :rtype: BllipParser )r)ÚclsÚ model_dirrrZparser_model_dirZreranker_features_filenameZreranker_weights_filenamerrr Úfrom_unified_model_dirÙsz"BllipParser.from_unified_model_dir)NNNNN)NN) Ú__name__Ú __module__Ú __qualname__Ú__doc__rrr#Ú classmethodr&rrrr rus )c Csddlm}|dƒj}tdƒt |¡}tdƒd ¡}d ¡}d ¡}x\|||fD]N}td d  |¡ƒyt|  |¡ƒ}t|ƒWqTt k r td ƒYqTXqTWx*t |  |¡ƒD]\}} td || fƒq¶Wtd t|  ddg¡ƒƒtdt|  ddg¡ƒƒtdt|  ddg¡ƒƒdS)z8This assumes the Python module bllipparser is installed.r)Úfindzmodels/bllip_wsj_no_auxzLoading BLLIP Parsing models...zDone.z#British left waffles on Falklands .z"I saw the man with the telescope .z # ! ? : -z Sentence: %rú z(parse failed)z parse %d: %szforcing 'tree' to be 'NN':)ÚAN)ÚtreeZNNz.forcing 'A' to be 'DT' and 'tree' to be 'NNP':)r.ZDT)r/ÚNNPzforcing 'A' to be 'NNP':)r.r0)r/NN) Z nltk.datar,ÚpathÚprintrr&ÚsplitÚjoinÚnextrÚ StopIterationrr#) r,r%ZbllipZ sentence1Z sentence2Zfail1rr/rrrrr Údemos4    r7cCs8ddlm}y tƒWntk r2|dƒ‚YnXdS)Nr)ÚSkipTestzZdoctests from nltk.parse.bllip are skipped because the bllipparser module is not installed)Znoser8r r )Úmoduler8rrr Ú setup_module1s   r:)Z __future__rZnltk.parse.apirZ nltk.treerÚ__all__Z bllipparserrZbllipparser.RerankingParserrr r r rrrr7r:rrrr Ú s   H     0