ó ßüÚ\c@`s„dZddlmZmZmZddlZddlmZmZddl Z ddl Z ddl Z ddl m Z ddlZddlm Z ddlmZmZmZdd lmZd „Zd „Zd „Zddd „Zd„Zd„Zd„Zdd„Zdddd„d„Zdddddd„d„Z ddddd„Z!d„Z"d„Z#d„Z$d„Z%dS(s, This module contains utility functions to construct and manipulate counting data structures for frames. When performing statistical profiling we obtain many call stacks. We aggregate these call stacks into data structures that maintain counts of how many times each function in that call stack has been called. Because these stacks will overlap this aggregation counting structure forms a tree, such as is commonly visualized by profiling tools. We represent this tree as a nested dictionary with the following form: { 'identifier': 'root', 'description': 'A long description of the line of code being run.', 'count': 10 # the number of times we have seen this line 'children': { # callers of this line. Recursive dicts 'ident-b': {'description': ... 'identifier': 'ident-a', 'count': ... 'children': {...}}, 'ident-b': {'description': ... 'identifier': 'ident-b', 'count': ... 'children': {...}}} } i(tprint_functiontdivisiontabsolute_importN(t defaultdicttdeque(tsleepi(ttime(t format_timetcolor_oftparse_timedelta(tget_thread_identitycC`sB|dkrdSdj|jj|jjt|jjƒfƒSdS(sq A string identifier from a frame Strings are cheaper to use as indexes into dicts than tuples or dicts tNonet;N(R tjointf_codetco_namet co_filenametstrtco_firstlineno(tframe((s2lib/python2.7/site-packages/distributed/profile.pyt identifier+s    cC`sU|j}d|j|j|jf}tj|j|j|jƒjƒ}|d|S(s> Render a frame as a line for inclusion into a text traceback s File "%s", line %s, in %ss (RRtf_linenoRt linecachetgetlinet f_globalstlstrip(Rtcottexttline((s2lib/python2.7/site-packages/distributed/profile.pyt repr_frame<s $cC`sV|j}tj|j|j|jƒjƒ}i|jd6|jd6|jd6|d6S(Ntfilenametnamet line_numberR(RRRRRRRR(RRR((s2lib/python2.7/site-packages/distributed/profile.pyt info_frameDs $   c`s*|d k r,t‡fd†|Dƒƒr,tSˆj}|d k rŽ|d ksc|jjj|ƒ rŽt|ˆ|d|ƒ}|tkrŽtSntˆƒ}y|d|}WnGt k rõidd6t ˆƒd6id6|d6}||d|>> import sys, threading >>> ident = threading.get_ident() # replace with your thread of interest >>> frame = sys._current_frames()[ident] >>> state = {'children': {}, 'count': 0, 'description': 'root', ... 'identifier': 'root'} >>> process(frame, None, state) >>> state {'count': 1, 'identifier': 'root', 'description': 'root', 'children': {'...'}} c3`s$|]}ˆjjj|ƒVqdS(N(RRtendswith(t.0to(R(s2lib/python2.7/site-packages/distributed/profile.pys cststoptchildrenitcountt descriptionRiN( R tanytFalsetf_backRRR"tprocessRtKeyErrorR!(RtchildtstateR%tomittprevtidenttd((Rs2lib/python2.7/site-packages/distributed/profile.pyR,Os,(  "      cG`sõ|s tƒSd„|Dƒ}t|ƒdkrHtdt|ƒƒ‚nttƒ}x<|D]4}x+|dD]}||j|d|ƒqlWq[Wd„|jƒDƒ}td„|Dƒƒ}i|ddd6t |ƒd6|d 6|dd d 6S( s& Merge multiple frame states together cS`sh|]}|d’qS(R((R#targ((s2lib/python2.7/site-packages/distributed/profile.pys ‡s isExpected identifiers, got %sR&cS`s%i|]\}}t|Œ|“qS((tmerge(R#tktv((s2lib/python2.7/site-packages/distributed/profile.pys s cs`s|]}|dVqdS(R'N((R#R4((s2lib/python2.7/site-packages/distributed/profile.pys siR(R'R( tcreatetlent ValueErrorRRtlisttappendtitemstsumtdict(targstsR&R4R.R'((s2lib/python2.7/site-packages/distributed/profile.pyR5ƒs   ! cC`s<idd6id6dd6idd6dd6dd 6dd 6d 6S( NiR'R&trootRtRRR RR(((((s2lib/python2.7/site-packages/distributed/profile.pyR8™s cC`s@g}x&|r.|jt|ƒƒ|j}q W|ddd…S(sX Create a call text stack from a frame Returns ------- list of strings Niÿÿÿÿ(R<RR+(RtL((s2lib/python2.7/site-packages/distributed/profile.pyt call_stack¢s   g{®Gáz„?c `s$g‰g‰ g‰g‰ g‰g‰g‰ g‰g‰g‰g‰‡‡‡‡‡‡‡‡‡‡ ‡ ‡ ‡ f d†‰ ˆ |dddƒgˆ D]}djd|ƒ^q}i ˆd6ˆ d6ˆd6ˆ d 6gˆD]}|d^qÔd 6ˆd 6ˆd 6ˆd 6ˆd6ˆd6ˆd6ˆ d6|d6S(s Convert a profile state into data useful by Bokeh See Also -------- plot_figure distributed.bokeh.components.ProfilePlot c `sš|dsdSˆj|ƒˆ j|ƒˆj|ƒ||}ˆ j|ƒˆj|ƒˆ jt|dˆƒƒ|d}ˆj|dƒˆj|dƒˆj|dƒˆj|dƒ|d}y|d}Wntk rúˆjdƒn0X|d krˆjd ƒnˆjt|ƒƒ|||d}|} xQ|d jƒD]?\} } | d|}ˆ | | | ||d ƒ| |7} qSWdS( NR'R(RRR RRtgrays t lightgrayR&i(R<Rt IndexErrorRR=( R/tstartR%theighttwidthtdescR2tfntdeltatxRR.( tcolorst filenamestheightst line_numberstlinestnamestprofile_intervaltstartststateststopsttimesttraversetwidths(s2lib/python2.7/site-packages/distributed/profile.pyR[Ås8           iis{:.2f}%idtlefttrighttbottomRKttoptcolorRXRRR RRt percentage(tformat(R/RVtwt percentagesRO(( RPRQRRRSRTRURVRWRXRYRZR[R\s2lib/python2.7/site-packages/distributed/profile.pyt plot_data°s83&&t20mst2scC`stS(N(R*(((s2lib/python2.7/site-packages/distributed/profile.pytþRCc C`sÅt|ƒ}t|ƒ}tƒ}tƒ}x”|ƒsÀtƒ||krt|jtƒ|fƒtƒ}tƒ}nytjƒ|}Wntk rœdSXt|d|d|ƒt |ƒq-WdS(NR0( R R8RR<tsyst_current_framesR-R,R R( t thread_idtlogtintervaltcycleR0R%trecenttlastR((s2lib/python2.7/site-packages/distributed/profile.pyt_watchþs        iècC`stS(N(R*(((s2lib/python2.7/site-packages/distributed/profile.pyRiRCc C`s†|d krtƒ}ntd|ƒ}tjdtdddi|d6|d6|d6|d 6|d 6|d 6ƒ}t|_|jƒ|S( sH Gather profile information on a particular thread This starts a new thread to watch a particular thread and returns a deque that holds periodic profile information. Parameters ---------- thread_id: int interval: str Time per sample cycle: str Time per refreshing to a new profile state maxlen: int Passed onto deque, maximum number of periods omit: str Don't include entries that start with this filename stop: callable Function to call to see if we should stop Returns ------- deque tmaxlenttargetRtProfiletkwargsRlRnRoRmR0R%N( R R Rt threadingtThreadRrtTruetdaemonRI(RlRnRoRsR0R%Rmtthread((s2lib/python2.7/site-packages/distributed/profile.pytwatchs       c C`s(tƒ}|dkrd}ntj||fƒ}|dkrHd}n4tj||fƒd}|t|ƒkr|d}n|dkr£|dkr£t|ƒ}nD|dkr»t|ƒn|}gt||ƒD]} || ^qÑ}tt j d|ƒŒ} |s t ƒS|r$t| |ƒ} n| S(s Collect profile information from a sequence of profile states Parameters ---------- history: Sequence[Tuple[time, Dict]] A list or deque of profile states recent: dict The most recent accumulating state start: time stop: time iiN( RR tbisectt bisect_leftt bisect_rightR9R;trangeR5ttoolztpluckR8( thistoryRpRIR%tkeytnowtistarttistoptiistoptitprof((s2lib/python2.7/site-packages/distributed/profile.pyt get_profileIs&      &c K`súddlm}m}ddlm}d|krGtj|dƒ}n|d|ƒ}|dd|}|jdd d d d d d dddd|ƒ}d|_ d|_ |ddddƒ}|j |ƒt |j _t |j_t |j_||fS(s§ Plot profile data using Bokeh This takes the output from the function ``plot_data`` and produces a Bokeh figure See Also -------- plot_data i(tColumnDataSourcetfigure(t HoverToolRXtdatattoolsttapR]R^R`R_Rat line_colortblackt line_widthitsourcet point_policyt follow_mousettooltipss.
Name:  @name
Filename:  @filename
Line number:  @line_number
Line:  @line
Time:  @time
Percentage:  @width
N(tbokeh.plottingRŒRt bokeh.modelsRŽRtdissoctquadR tselection_glyphtnonselection_glypht add_toolsR*txaxistvisibletyaxistgrid( RRvRŒRRŽR•tfigtrthover((s2lib/python2.7/site-packages/distributed/profile.pyt plot_figuress2          cc`s'x |D]}|jrPn|VqWdS(N(t is_python(tframestentry((s2lib/python2.7/site-packages/distributed/profile.pyt_remove_py_stackºs  cC`s|s dS|jƒ}|r1t|||ƒ}nt|j|jƒ}djtt|jd|fƒƒ}y|d|}Wnft k råidd6idd6|jd6dd 6t|ƒd 6d 6id6|d 6}||d|R&iR'RRR RR(Ri( tpopt llprocessthextaddrtoffsetR tmapRRR-R (R©R.R/RR¯R2R3((s2lib/python2.7/site-packages/distributed/profile.pyR­Ás0 $    cC`sEddlm}||dtƒ}tt|ƒƒddd…}|S(s4 Collect low level stack information from thread id i(tget_thread_stackt show_pythonNiÿÿÿÿ(t stacktraceR²R*R;R«(ttidR²R©tllframes((s2lib/python2.7/site-packages/distributed/profile.pyt ll_get_stackïs(&t__doc__t __future__RRRR}t collectionsRRRRjRwRRRtmetricstutilsRRR t compatibilityR RRR!R R,R5R8RERfRrR|R‹R§R«R­R·(((s2lib/python2.7/site-packages/distributed/profile.pyts>        4   N0* G  .