ó ßüÚ\c@`sddlmZmZmZddlmZddlZddlZddlZddl m Z m Z ddl m Z ddlmZejeƒZZdefd „ƒYZd efd „ƒYZd efd „ƒYZeƒZdaejƒZd„Zed„ZdS(i(tprint_functiontdivisiontabsolute_import(tdequeNi(tPY2tPYPY(t thread_time(t format_bytest ThrottledGCcB`s)eZdZdddd„Zd„ZRS(s Wrap gc.collect to protect against excessively repeated calls. Allows to run throttled garbage collection in the workers as a countermeasure to e.g.: https://github.com/dask/zict/issues/19 collect() does nothing when repeated calls are so costly and so frequent that the thread would spend more than max_in_gc_frac doing GC. warn_if_longer is a duration in seconds (10s by default) that can be used to log a warning level message whenever an actual call to gc.collect() lasts too long. gš™™™™™©?icC`sF||_||_tƒ|_d|_|dk r9|nt|_dS(Ni(tmax_in_gc_fractwarn_if_longerRt last_collecttlast_gc_durationtNonet_loggertlogger(tselfR R R((s5lib/python2.7/site-packages/distributed/utils_perf.pyt__init__s     cC`sÛd}tƒ}t||j|ƒ}|j||jkr¾|jjd|ƒtjƒ||_ttƒ||ƒ|_|j|j kr¥|jj d|jƒq×|jjd|jƒn|jjd|j|ƒdS(Ngíµ ÷Æ°>s9Calling gc.collect(). %0.3fs elapsed since previous call.s¨gc.collect() took %0.3fs. This is usually a sign that the some tasks handle too many Python objects at the same time. Rechunking the work into smaller tasks might help.sgc.collect() took %0.3fssNgc.collect() lasts %0.3fs but only %0.3fs elapsed since last call: throttling.( RtmaxR R R RtdebugtgctcollectR twarning(Rt MIN_RUNTIMEt collect_starttelapsed((s5lib/python2.7/site-packages/distributed/utils_perf.pyR%s$        N(t__name__t __module__t__doc__R RR(((s5lib/python2.7/site-packages/distributed/utils_perf.pyRs tFractionalTimercB`sJeZdZdZed„Zd„Zd„Zd„Ze d„ƒZ RS(s  An object that measures runtimes, accumulates them and computes a running fraction of the recent runtimes over the corresponding elapsed time. geÍÍAcC`sI||_||_tƒ|_tƒ|_d|_d|_d|_dS(N( t_timert _n_samplesRt _start_stopst _durationsR t _cur_startt _running_sumt_running_fraction(Rt n_samplesttimer((s5lib/python2.7/site-packages/distributed/utils_perf.pyRPs      c C`s>|j}|j}||ks8|r<||ddkr<dSt|||jƒ}|j||fƒ|j|ƒt|ƒ}|t|ƒks—t‚||jkr:|jdkrÜ||jksÊt‚t |ƒ|_q:|j ƒ\}}|j ƒ} |j|| 7_||kr:|j|||j|_ q:ndS(Niÿÿÿÿi( R R!tinttMULTtappendtlentAssertionErrorRR#R tsumtpopleftR$( Rtstarttstopt start_stopst durationstdurationtnt old_starttold_stopt old_duration((s5lib/python2.7/site-packages/distributed/utils_perf.pyt_add_measurementYs$  &    cC`s(|jdkst‚|jƒ|_dS(N(R"R R+R(R((s5lib/python2.7/site-packages/distributed/utils_perf.pyt start_timingtscC`sD|jƒ}|j}d|_|dk s0t‚|j||ƒdS(N(RR"R R+R7(RR/R.((s5lib/python2.7/site-packages/distributed/utils_perf.pyt stop_timingxs    cC`s|jS(N(R$(R((s5lib/python2.7/site-packages/distributed/utils_perf.pytrunning_fractions( RRRR(RRR7R8R9tpropertyR:(((s5lib/python2.7/site-packages/distributed/utils_perf.pyRGs   t GCDiagnosiscB`s_eZdZdZdd d„Zd„Zd„Zed„ƒZd „Z d „Z d „Z RS( sß An object that hooks itself into the gc callbacks to collect timing and memory statistics, and log interesting info. Don't instantiate this directly except for tests. Instead, use the global instance. igš™™™™™¹?i g€„.AcC`s||_||_t|_dS(N(t_warn_over_fract_info_over_rss_wintFalset_enabled(Rtwarn_over_fractinfo_over_rss_win((s5lib/python2.7/site-packages/distributed/utils_perf.pyRs  cC`s¬ts trdS|j s t‚td|jƒ|_yddl}Wntk rad|_ nX|j ƒ|_ |j }|t jkst‚t jj|ƒt|_dS(NR%i(RRR@R+Rt N_SAMPLESt_fractional_timertpsutilt ImportErrorR t_proctProcesst _gc_callbackRt callbacksR)tTrue(RREtcb((s5lib/python2.7/site-packages/distributed/utils_perf.pytenable”s    cC`s?ts trdS|jst‚tjj|jƒt|_dS(N( RRR@R+RRJtremoveRIR?(R((s5lib/python2.7/site-packages/distributed/utils_perf.pytdisable¦s  cC`s|jS(N(R@(R((s5lib/python2.7/site-packages/distributed/utils_perf.pytenabled­scC`s|jƒ|S(N(RM(R((s5lib/python2.7/site-packages/distributed/utils_perf.pyt __enter__±s cG`s|jƒdS(N(RO(Rtargs((s5lib/python2.7/site-packages/distributed/utils_perf.pyt__exit__µscC`s>|ddkrdS|jdk r8|jjƒj}nd}|dkrd|jjƒ||_dS|dksvt‚|jjƒ|jj }|dk rË||j krËt j dd|d|j ƒn|j|}||j krt jdt|ƒ|d t|j ƒƒn|d dkr:t j d |d ƒndS( Nt generationiiR.R/sFfull garbage collections took %d%% CPU time recently (threshold: %d%%)idsLfull garbage collection released %s from %d reference cycles (threshold: %s)t collectedt uncollectablesHgarbage collector couldn't collect %d objects, please look in gc.garbage(RGR t memory_infotrssRDR8t_gc_rss_beforeR+R9R:R=RRR>tinfoR(RtphaseRZRXtfract rss_saved((s5lib/python2.7/site-packages/distributed/utils_perf.pyRI¸s8       gÐcA( RRRRCRRMROR;RPRQRSRI(((s5lib/python2.7/site-packages/distributed/utils_perf.pyR<„s    cC`sSts trdSt7tdkr0tjƒntjs?t‚td7aWdQXdS(s, Ask to enable global GC diagnosis. Nii(RRt_gc_diagnosis_lockt_gc_diagnosis_userst _gc_diagnosisRMRPR+(((s5lib/python2.7/site-packages/distributed/utils_perf.pytenable_gc_diagnosisæs   cC`s{ts trdSt_tdkrqtd8atdkrFtjƒqq|r_tjƒdaqqtjsqt‚nWdQXdS(s- Ask to disable global GC diagnosis. Nii(RRR^R_R`RORPR+(tforce((s5lib/python2.7/site-packages/distributed/utils_perf.pytdisable_gc_diagnosisõs       (t __future__RRRt collectionsRRtloggingt threadingt compatibilityRRtmetricsRtutilsRt getLoggerRRRtobjectRRR<R`R_tLockR^RaR?Rc(((s5lib/python2.7/site-packages/distributed/utils_perf.pyts    7=]