ó ­V]c@sodZdZddlZddlmZddlmZddlZddlmZmZddlm Z ddl Z ddl Z ddl Z ddl Z ddlZddlZddlZddlZd„Zed „Zd efd „ƒYZd „Zd ZdZdd„Zdd„Zdd„Zdd„Zddd„ZedkrkeejjƒƒndS(s=Diagnostic functions, mainly for use when doing tech support.tMITi˙˙˙˙N(tStringIO(t HTMLParser(t BeautifulSoupt __version__(tbuilder_registryc CsVdtGHdtjGHdddg}xK|D]C}x:tjD]}||jkr;Pq;q;W|j|ƒd|GHq+Wd|krŰ|jdƒy2dd lm }d d j t t |j ƒƒGHWqŰtk r×}d GHqŰXnd|kr!ydd l}d|jGHWq!tk r}dGHq!Xnt|dƒr?|jƒ}nˆ|jdƒs]|jdƒrod|GHdGHd SyCtjj|ƒrąd|GHt|ƒ}|jƒ}Wd QXnWntk rĹnXHxˆ|D]€}d|GHt} yt|d|ƒ} t} Wn&tk r'}d|GHtjƒnX| rEd|GH| jƒGHnddGHqÎWd S(s/Diagnostic suite for isolating common problems.s'Diagnostic running on Beautiful Soup %ssPython version %ss html.parserthtml5libtlxmls;I noticed that %s is not installed. Installing it may help.slxml-xmli˙˙˙˙(tetreesFound lxml version %st.s.lxml is not installed or couldn't be imported.NsFound html5lib version %ss2html5lib is not installed or couldn't be imported.treadshttp:shttps:s<"%s" looks like a URL. Beautiful Soup is not an HTTP client.spYou need to use some other library to get the document behind the URL, and feed that document to Beautiful Soup.s7"%s" looks like a filename. Reading data from the file.s#Trying to parse your markup with %stfeaturess%s could not parse the markup.s#Here's what %s did with the markup:t-iP(RtsystversionRtbuildersR tremovetappendRRtjointmaptstrt LXML_VERSIONt ImportErrorRthasattrR t startswithtostpathtexiststopent ValueErrortFalseRtTruet Exceptiont tracebackt print_exctprettify( tdatat basic_parserstnametbuilderRteRtfptparsertsuccesstsoup((s+lib/python2.7/site-packages/bs4/diagnose.pytdiagnosesd       "            cKs[ddlm}xD|jt|ƒd||D]$\}}d||j|jfGHq/WdS(s—Print out the lxml events that occur during parsing. This lets you see how lxml parses a document when no Beautiful Soup code is running. i˙˙˙˙(Rthtmls %s, %4s, %sN(RRt iterparseRttagttext(R$R.tkwargsRteventtelement((s+lib/python2.7/site-packages/bs4/diagnose.pyt lxml_traceYs+tAnnouncingParsercBsheZdZd„Zd„Zd„Zd„Zd„Zd„Zd„Z d„Z d „Z d „Z RS( s?Announces HTMLParser parse events, without doing anything else.cCs |GHdS(N((tselfts((s+lib/python2.7/site-packages/bs4/diagnose.pyt_pfscCs|jd|ƒdS(Ns%s START(R9(R7R&tattrs((s+lib/python2.7/site-packages/bs4/diagnose.pythandle_starttagiscCs|jd|ƒdS(Ns%s END(R9(R7R&((s+lib/python2.7/site-packages/bs4/diagnose.pyt handle_endtaglscCs|jd|ƒdS(Ns%s DATA(R9(R7R$((s+lib/python2.7/site-packages/bs4/diagnose.pyt handle_dataoscCs|jd|ƒdS(Ns %s CHARREF(R9(R7R&((s+lib/python2.7/site-packages/bs4/diagnose.pythandle_charrefrscCs|jd|ƒdS(Ns %s ENTITYREF(R9(R7R&((s+lib/python2.7/site-packages/bs4/diagnose.pythandle_entityrefuscCs|jd|ƒdS(Ns %s COMMENT(R9(R7R$((s+lib/python2.7/site-packages/bs4/diagnose.pythandle_commentxscCs|jd|ƒdS(Ns%s DECL(R9(R7R$((s+lib/python2.7/site-packages/bs4/diagnose.pyt handle_decl{scCs|jd|ƒdS(Ns%s UNKNOWN-DECL(R9(R7R$((s+lib/python2.7/site-packages/bs4/diagnose.pyt unknown_decl~scCs|jd|ƒdS(Ns%s PI(R9(R7R$((s+lib/python2.7/site-packages/bs4/diagnose.pyt handle_pis( t__name__t __module__t__doc__R9R;R<R=R>R?R@RARBRC(((s+lib/python2.7/site-packages/bs4/diagnose.pyR6cs         cCstƒ}|j|ƒdS(sŁPrint out the HTMLParser events that occur during parsing. This lets you see how HTMLParser parses a document when no Beautiful Soup code is running. N(R6tfeed(R$R*((s+lib/python2.7/site-packages/bs4/diagnose.pythtmlparser_trace„s taeioutbcdfghjklmnpqrstvwxyzicCsSd}xFt|ƒD]8}|ddkr2t}nt}|tj|ƒ7}qW|S(s#Generate a random word-like string.tii(tranget _consonantst_vowelstrandomtchoice(tlengthR8titt((s+lib/python2.7/site-packages/bs4/diagnose.pytrwords icCsdjd„t|ƒDƒƒS(s'Generate a random sentence-like string.t css'|]}ttjddƒƒVqdS(ii N(RTROtrandint(t.0RR((s+lib/python2.7/site-packages/bs4/diagnose.pys s(RRL(RQ((s+lib/python2.7/site-packages/bs4/diagnose.pyt rsentence›sičcCsëdddddddg}g}x˛t|ƒD]¤}tjdd ƒ}|dkrutj|ƒ}|jd |ƒq.|d krŁ|jttjd d ƒƒƒq.|d kr.tj|ƒ}|jd|ƒq.q.Wddj|ƒdS(s+Randomly generate an invalid HTML document.tptdivtspanRRtbtscriptttableiis<%s>iiisss s(RLRORVRPRRXR(t num_elementst tag_namestelementsRRRPttag_name((s+lib/python2.7/site-packages/bs4/diagnose.pytrdocŸs  " i †c CsSdtGHt|ƒ}dt|ƒGHxœdddgddgD]‚}t}y1tjƒ}t||ƒ}tjƒ}t}Wn&tk r˘}d|GHtj ƒnX|r=d|||fGHq=q=Wd d l m }tjƒ}|j |ƒtjƒ}d ||GHd d l } | jƒ}tjƒ}|j|ƒtjƒ}d ||GHd S(s.Very basic head-to-head performance benchmark.s1Comparative parser benchmark on Beautiful Soup %ss3Generated a large invalid HTML document (%d bytes).RR.Rs html.parsers%s could not parse the markup.s"BS4+%s parsed the markup in %.2fs.i˙˙˙˙(Rs$Raw lxml parsed the markup in %.2fs.Ns(Raw html5lib parsed the markup in %.2fs.(RRctlenRttimeRRR R!R"RRtHTMLRRtparse( R_R$R*R+taR,R\R(RR((s+lib/python2.7/site-packages/bs4/diagnose.pytbenchmark_parsersąs4               RcCs‚tjƒ}|j}t|ƒ}tdtd|d|ƒ}tjd|||ƒtj |ƒ}|j dƒ|j ddƒdS(Ntbs4R$R*sbs4.BeautifulSoup(data, parser)t cumulatives _html5lib|bs4i2( ttempfiletNamedTemporaryFileR&RctdictRjtcProfiletrunctxtpstatstStatst sort_statst print_stats(R_R*t filehandletfilenameR$tvarststats((s+lib/python2.7/site-packages/bs4/diagnose.pytprofileŃs    t__main__( RFt __license__RoRRRjRRt bs4.builderRRRqRORlReR!R R-RR5R6RHRNRMRTRXRcRiRyRDtstdinR (((s+lib/python2.7/site-packages/bs4/diagnose.pyts8           C !