`]c@sddlmZmZddlZddlZddlmZddlmZddl m Z ddl m Z dej fdYZd ej fd YZdS( i(tprint_functiontunicode_literalsN(tclosing(tdata(tSnowballStemmer(t PorterStemmert SnowballTestcBs5eZdZdZdZdZdZRS(cCstdt}|jddks*t|jddksEt|jddks`t|jddks{t|jd dkst|jd dkst|jd dkst|jd d kst|jd d ksttdt}|jd dks,t|jddksGt|jddksbttd}|jddkst|jddkst|jddkst|jddkst|jddkstdS(u this unit testing for test the snowball arabic light stemmer this stemmer deals with prefixes and suffixes uarabicu&الْعَرَبِــــــيَّةuعربuالعربيةu فقالواuقالuالطالباتuطالبuفالطالباتuوالطالباتuالطالبونu اللذانuمنuاللذuالكلماتuكلمN(RtTruetstemtAssertionErrortFalse(tselft ar_stemmer((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyt test_arabics( cCs+td}|jddks'tdS(Nurussianuавантненькаяuавантненьк(RRR (R tstemmer_russian((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyt test_russian+s cCstd}tddt}|jddks9t|jddksTt|jddksot|jddkstdS(Nugermantignore_stopwordsu Schränkeuschrankukeinenukein(RRRR (R tstemmer_germantstemmer_german2((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyt test_german/s  cCsFtd}|jddks't|jddksBtdS(Nuspanishu Visionadouvisionualgueualgu(RRR (R tstemmer((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyt test_spanish9s cCs+td}|jddks'tdS(Nuenglishuy'suy(RRR (R R((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyttest_short_strings_bugAs (t__name__t __module__R RRRR(((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyR s    t PorterTestcBs>eZdZdZdZdZdZdZRS(cCs>ttjdjdd}|jjSWdQXdS(Nu*stemmers/porter_test/porter_vocabulary.txttencodinguutf-8(RRtfindtopentreadt splitlines(R tfp((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyt _vocabularyGs  cCsptd|}xZt|j|D]C\}}|j|}||ks%td||||fq%WdS(Ntmodeu*%s should stem to %s in %s mode but got %s(RtzipR RR (R t stemmer_modetexpected_stemsRtwordt true_stemtour_stem((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyt_test_against_expected_outputOs "cCsMttjdjdd&}|jtj|jjWdQXdS(uTests all words from the test vocabulary provided by M Porter The sample vocabulary and output were sourced from: http://tartarus.org/martin/PorterStemmer/voc.txt http://tartarus.org/martin/PorterStemmer/output.txt and are linked to from the Porter Stemmer algorithm's homepage at http://tartarus.org/martin/PorterStemmer/ u-stemmers/porter_test/porter_martin_output.txtRuutf-8N( RRRRR(RtMARTIN_EXTENSIONSRR(R R((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyttest_vocabulary_martin_modeXs  cCsMttjdjdd&}|jtj|jjWdQXdS(Nu+stemmers/porter_test/porter_nltk_output.txtRuutf-8( RRRRR(RtNLTK_EXTENSIONSRR(R R((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyttest_vocabulary_nltk_modeks  cCsttjdjdd&}|jtj|jjWdQX|jtjtjdjddjjdS(Nu/stemmers/porter_test/porter_original_output.txtRuutf-8( RRRRR(RtORIGINAL_ALGORITHMRR(R R((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyttest_vocabulary_original_modeus cCs"tjddkstdS(uTest for bug https://github.com/nltk/nltk/issues/1581 Ensures that 'oed' can be stemmed without throwing an error. uoeduoN(RRR (R ((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyt test_oed_bugs(RRR R(R*R,R.R/(((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyRFs    (t __future__RRtostunittestt contextlibRtnltkRtnltk.stem.snowballRtnltk.stem.porterRtTestCaseRR(((s7lib/python2.7/site-packages/nltk/test/unit/test_stem.pyts  9