B <[? @sdZddlmZy(ddlZddlmZddlmZdZWne e fk rTdZYnXdd d gZ Gd d d eZ Gd dde ZGddde ZdS)z@Lexical analysis of formal languages (i.e. code) using Pygments.)ApplicationErrorN)get_lexer_by_name)_get_ttype_classTFtokentextc@s eZdZdS) LexerErrorN)__name__ __module__ __qualname__r r ;lib/python3.7/site-packages/docutils/utils/code_analyzer.pyrsrc@s*eZdZdZd ddZddZddZd S) LexeraParse `code` lines and yield "classified" tokens. Arguments code -- string of source code to parse, language -- formal language the code is written in, tokennames -- either 'long', 'short', or '' (see below). Merge subsequent tokens of the same token-type. Iterating over an instance yields the tokens as ``(tokentype, value)`` tuples. The value of `tokennames` configures the naming of the tokentype: 'long': downcased full token type name, 'short': short name defined by pygments.token.STANDARD_TYPES (= class argument used in pygments html output), 'none': skip lexical analysis. shortcCsr||_||_||_d|_|dks(|dkr,dSts8tdyt|j|_Wn$tjj k rltd|YnXdS)zE Set up a lexical analyzer for `code` in `language`. N)rrZnonez0Cannot analyze code. Pygments package not found.z6Cannot analyze code. No Pygments lexer found for "%s".) codelanguage tokennameslexer with_pygmentsrrpygmentsutilZ ClassNotFound)selfrrrr r r __init__0szLexer.__init__ccstt|}t|\}}x6|D].\}}||kr4||7}q||fV||}}qW|drb|dd}|rp||fVdS)zrMerge subsequent tokens of same token-type. Also strip the final newline (added by pygments).  N)iternextendswith)rtokensZlasttypeZlastvalttypevaluer r r mergeHs     z Lexer.mergeccs|jdkrg|jfVdSt|j|j}xV||D]H\}}|jdkr\t|d}n t |g}dd|D}||fVq6WdS)z7Parse self.code and yield "classified" tokens. NZlong.cSsg|]}|tkr|qSr )unstyled_tokens).0clsr r r fsz"Lexer.__iter__..) rrrZlexr!rstrlowersplitr)rrZ tokentyper classesr r r __iter__Zs    zLexer.__iter__N)r)r r r __doc__rr!r+r r r r rs rc@s eZdZdZddZddZdS) NumberLinesatInsert linenumber-tokens at the start of every code line. Arguments tokens -- iterable of ``(classes, value)`` tuples startline -- first line number endline -- last line number Iterating over an instance yields the tokens with a ``(['ln'], '')`` token added for every code line. Multi-line tokens are splitted.cCs"||_||_dtt||_dS)Nz%%%dd )r startlinelenr'fmt_str)rrr.Zendliner r r rwszNumberLines.__init__ccs|j}dg|j|fVxh|jD]^\}}|d}x<|ddD],}||dfV|d7}dg|j|fVq@W||dfVq WdS)NZlnrr)r.r0rr))rlinenorr linesliner r r r+}s zNumberLines.__iter__N)r r r r,rr+r r r r r-js r-)r,ZdocutilsrrZpygments.lexersrZpygments.formatters.htmlrr ImportError SyntaxErrorr#robjectrr-r r r r s    N