ó •‰]c@sXdZddlZdefd„ƒYZdefd„ƒYZdefd„ƒYZdS( sÇ This module contains a tokenizer for Excel formulae. The tokenizer is based on the Javascript tokenizer found at http://ewbi.blogs.com/develops/2004/12/excel_formula_p.html written by Eric Bachtal iÿÿÿÿNtTokenizerErrorcBseZdZRS(s$Base class for all Tokenizer errors.(t__name__t __module__t__doc__(((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyR st TokenizercBsßeZdZejdƒZejdƒZiejdƒd6ejdƒd6ZdZdZ d„Z d„Z d„Z d„Z d„Zd„Zd„Zd„Zd„Zd„Zd„Zdd„Zd„Zd„ZRS( sf A tokenizer for Excel worksheet formulae. Converts a unicode string representing an Excel formula (in A1 notation) into a sequence of `Token` objects. `formula`: The unicode string to tokenize Tokenizer defines a method `._parse()` to parse the formula into tokens, which can then be accessed through the `.items` attribute. s^[1-9](\.[0-9]+)?[Ee]$s[ \n]+s"(?:[^"]*"")*[^"]*"(?!")t"s'(?:[^']*'')*[^']*'(?!')t's#NULL!s#DIV/0!s#VALUE!s#REF!s#NAME?s#NUM!s#N/As #GETTING_DATAs,;}) +-*/^&=><%cCs;||_g|_g|_d|_g|_|jƒdS(Ni(tformulatitemst token_stacktoffsetttokent_parse(tselfR((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyt__init__.s      c Cs¹|jr dS|jsdS|jddkr?|jd7_n#|jjt|jtjƒƒdSd|jfd|jfd|jfd|j fd |j fd |j fd |j fd |j fd |j ff }i}x-|D]%\}}|jtj||ƒƒqáWxž|jt|jƒkrª|jƒr7q n|j|j}||jkrc|jƒn||krˆ|j||ƒ7_q |jj|ƒ|jd7_q W|jƒdS(s5Populate self.items with the tokens from the formula.Nit=is"'t[t#t s s +-*/^&=><%s{(s)}s;,(R RRtappendtTokentLITERALt _parse_stringt_parse_bracketst _parse_errort_parse_whitespacet_parse_operatort _parse_openert _parse_closert_parse_separatortupdatetdicttfromkeystlentcheck_scientific_notationt TOKEN_ENDERSt save_tokenR (R t consumerst dispatchertcharstconsumert curr_char((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyR 7s>             cCsé|jddƒ|j|j}|d ks2t‚|j|}|j|j|jƒ}|d kr˜|dkrvdnd}td||jfƒ‚n|jdƒ}|dkrÏ|j j t j |ƒƒn|j j |ƒt|ƒS( s¹ Parse a "-delimited string or '-delimited link. The offset must be pointing to either a single quote ("'") or double quote ('"') character. The strings are parsed according to Excel rules where to escape the delimiter you just double it up. E.g., "abc""def" in Excel is parsed as 'abc"def' in Python. Returns the number of characters matched. (Does not update self.offset) t can_followt:RRtstringtlinks-Reached end of formula while parsing %s in %si(RRN(tassert_empty_tokenRR tAssertionErrortSTRING_REGEXEStmatchtNoneRtgroupRRRt make_operandR R!(R tdelimtregexR1tsubtype((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyR_s    cCs|j|jdkst‚gtjd|j|jƒD]}|jƒdf^q9}gtjd|j|jƒD]}|jƒdf^qt}d}xft||ƒD]T\}}||7}|dkr©|d}|jj|j|j|j|!ƒ|Sq©Wt d|jƒ‚dS( sœ Consume all the text between square brackets []. Returns the number of characters matched. (Does not update self.offset) Rs\[is\]iÿÿÿÿisEncountered unmatched '[' in %sN( RR R/tretfinditertstarttsortedR RR(R tttleftstrightst open_counttidxt open_closet outer_right((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyR}s88    cCs¹|jddƒ|j|jdks,t‚|j|j}xZ|jD]O}|j|ƒrF|jjtj dj |j ƒ|ƒƒ|j 2t |ƒSqFWt d|j|jfƒ‚dS(sà Consume the text following a '#' as an error. Looks for a match in self.ERROR_CODES and returns the number of characters matched. (Does not update self.offset) R*t!Rts)Invalid error code at position %d in '%s'N(R.RR R/t ERROR_CODESt startswithRRRR4tjoinR R!R(R t subformulaterr((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyR–s)cCsb|j|jdkst‚|jjt|j|jtjƒƒ|jj|j|jƒj ƒS(s† Consume a string of consecutive spaces. Returns the number of spaces found. (Does not update self.offset). Rs (Rs ( RR R/RRRtWSPACEt WSPACE_RER1tend(R ((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyRªs&cCsu|j|j|jd!d krT|jjt|j|j|jd!tjƒƒdS|j|j}|dksvt‚|dkr—tdtjƒ}nÊ|dkr¸t|tjƒ}n©|jsÖt|tjƒ}n‹t d„t |jƒDƒd ƒ}|o1|j tj kp1|jtjkp1|jtjk}|rOt|tjƒ}nt|tjƒ}|jj|ƒd S( s Consume the characters constituting an operator. Returns the number of characters consumed. (Does not update self.offset) is>=s<=s<>s %*/^&=><+-t%s*/^&=>Ísi(s>=s<=s<>N(RR RRRtOP_INR/tOP_POSTtOP_PREtnexttreversedR2R7tCLOSERNtOPERAND(R R)R tprevtis_infix((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyRµs0       cCs¹|j|jdkst‚|j|jdkrN|jƒtjdƒ}nG|jr†dj|jƒd}|j2tj|ƒ}ntjdƒ}|jj |ƒ|j j |ƒdS(s‰ Consumes a ( or { character. Returns the number of characters consumed. (Does not update self.offset) t(t{RDi(RZR[( RR R/R.Rt make_subexpR RGRRR (R R t token_value((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyRÛs  cCst|j|jdkst‚|jjƒjƒ}|j|j|jkr`td|jƒ‚n|jj |ƒdS(s‰ Consumes a } or ) character. Returns the number of characters consumed. (Does not update self.offset) t)t}sMismatched ( and { pair in '%s'i(R^R_( RR R/R tpopt get_closertvalueRRR(R R ((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyRñscCsÁ|j|j}|dks"t‚|dkr@tjdƒ}nmy|jdj}Wn#tk rytdtjƒ}n4X|tj kržtdtjƒ}ntjdƒ}|j j |ƒdS(s‰ Consumes a ; or , character. Returns the number of characters consumed. (Does not update self.offset) t;t,iÿÿÿÿi(RcRd( RR R/Rtmake_separatorR RNt IndexErrorRQtPARENRR(R R)R ttop_type((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyRs  cCsv|j|j}|dkrrt|jƒdkrr|jjdj|jƒƒrr|jj|ƒ|jd7_tSt S(s¾ Consumes a + or - character if part of a number in sci. notation. Returns True if the character was consumed and self.offset was updated, False otherwise. s+-iRD( RR R!R tSN_RER1RGRtTruetFalse(R R)((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyR"s cCs?|jr;|jd|kr;td|j|jfƒ‚ndS(s: Ensure that there's no token currently being parsed. Or if there is a token being parsed, it must end with a character in can_follow. If there are unconsumed token contents, it means we hit an unexpected token transition. In this case, we raise a TokenizerError iÿÿÿÿs+Unexpected character at position %d in '%s'N(R RR R(R R*((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyR.+s cCs<|jr8|jjtjdj|jƒƒƒ|j2ndS(s9If there's a token being parsed, add it to the item list.RDN(R RRRR4RG(R ((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyR$;s %cCsR|js dS|jdjtjkr4|jdjSddjd„|jDƒƒS(s+Convert the parsed tokens back to a string.RDiRcss|]}|jVqdS(N(Rb(ROR ((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pys Gs(RRNRRRbRG(R ((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pytrenderAs  (s#NULL!s#DIV/0!s#VALUE!s#REF!s#NAME?s#NUM!s#N/As #GETTING_DATA((RRRR8tcompileRiRKR0RER#RR RRRRRRRRR"R.R$Rl(((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyRs.  (    &      RcBsÝeZdZdddgZdZdZdZdZdZd Z d Z d Z d Z d Z dd„ZdZdZdZdZdZd„Zed„ƒZdZdZeed„ƒZd„ZdZdZed„ƒZRS(s) A token in an Excel formula. Tokens have three attributes: * `value`: The string value parsed that led to this token * `type`: A string identifying the type of token * `subtype`: A string identifying subtype of the token (optional, and defaults to "") RbRNR7RRWtFUNCtARRAYRgtSEPsOPERATOR-PREFIXsOPERATOR-INFIXsOPERATOR-POSTFIXs WHITE-SPACERDcCs||_||_||_dS(N(RbRNR7(R Rbttype_R7((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyRes  tTEXTtNUMBERtLOGICALtERRORtRANGEcCsdj|j|j|jƒS(Nu {0} {1} {2}:(tformatRNR7Rb(R ((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyt__repr__wscCs•|jdƒr|j}ng|jdƒr6|j}nL|dkrN|j}n4yt|ƒ|j}Wntk r|j}nX|||j|ƒS(sCreate an operand token.RRtTRUEtFALSE(RyRz( RFRrRuRttfloatRst ValueErrorRvRW(tclsRbR7((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyR4zs        tOPENRVcCs§|dd kst‚|r@tjd|ƒs4t‚tj}n9|dkrXtj}n!|dkrptj}n tj}|d krŽ|jn|j}||||ƒS( s• Create a subexpression token. `value`: The value of the token `func`: If True, force the token to be of type FUNC iÿÿÿÿR[R_RZR^s.+\(|\)s{}s()s)}(R[R_RZR^( R/R8R1RRnRoRgRVR~(R}RbtfuncRqR7((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyR\—s       cCs||j|j|j|jfks't‚|j|jks?t‚|j|jkrWdnd}|j|d|j|jkƒS(s6Return a closing token that matches this token's type.R_R^R(RNRnRoRgR/R7R~R\(R Rb((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyRa­s'tARGtROWcCsC|dkst‚|dkr'|jn|j}|||j|ƒS(sCreate a separator tokenRdRc(RdRc(R/R€RRp(R}RbR7((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyRe¿s(RRRt __slots__RRWRnRoRgRpRSRQRRRJRRrRsRtRuRvRxt classmethodR4R~RVRkR\RaR€RRe(((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyRJs8    (RR8t ExceptionRtobjectRR(((s9lib/python2.7/site-packages/openpyxl/formula/tokenizer.pyts  ÿ;