ó ßüÚ\c@sddlZddlZddlmZddlZddlmZddlmZm Z m Z m Z ej e ƒZdefd„ƒYZdS( iÿÿÿÿN(trefi(tAdaptivei(t format_bytestPeriodicCallbackt log_errorstignoringtClustercBsYeZdZd„Zed„ƒZed„ƒZd„Zd„Zd„Z d„Z RS(sê Superclass for cluster objects This expects a local Scheduler defined on the object. It provides common methods and an IPython widget display. Clusters inheriting from this class should provide the following: 1. A local ``Scheduler`` object at ``.scheduler`` 2. scale_up and scale_down methods as defined below:: def scale_up(self, n: int): ''' Brings total worker count up to ``n`` ''' def scale_down(self, workers: List[str]): ''' Close the workers with the given addresses ''' This will provide a general ``scale`` method as well as an IPython widget for display. Examples -------- >>> from distributed.deploy import Cluster >>> class MyCluster(cluster): ... def scale_up(self, n): ... ''' Bring the total worker count up to n ''' ... pass ... def scale_down(self, workers): ... ''' Close the workers with the given addresses ''' ... pass >>> cluster = MyCluster() >>> cluster.scale(5) # scale manually >>> cluster.adapt(minimum=1, maximum=100) # scale automatically See Also -------- LocalCluster: a simple implementation with local workers cKsmttƒ|jjƒWdQXt|dƒs;i|_n|jj|ƒt|j||j|_|jS(s¿ Turn on adaptivity For keyword arguments see dask.distributed.Adaptive Examples -------- >>> cluster.adapt(minimum=0, maximum=10, interval='500ms') Nt_adaptive_options( RtAttributeErrort _adaptivetstopthasattrRtupdateRt scheduler(tselftkwargs((s9lib/python2.7/site-packages/distributed/deploy/cluster.pytadapt7s  cCs |jjS(N(R taddress(R((s9lib/python2.7/site-packages/distributed/deploy/cluster.pytscheduler_addressHscCsgtjjdƒ}|jjjdƒdjdƒd}|jjdj}|jd|d|t j S( Nsdistributed.dashboard.links://it:itbokehthosttport( tdasktconfigtgetR RtsplittservicesRtformattostenviron(RttemplateRR((s9lib/python2.7/site-packages/distributed/deploy/cluster.pytdashboard_linkLs&cCsµtƒ¦|t|jjƒkr>|jjj|j|ƒnm|jjdt|jjƒ|ƒ}tj d|ƒ|jjj|jj d|ƒ|jjj|j |ƒWdQXdS(s5 Scale cluster to n workers Parameters ---------- n: int Target number of workers Example ------- >>> cluster.scale(10) # scale cluster to ten workers See Also -------- Cluster.scale_up Cluster.scale_down tnsClosing workers: %stworkersN( RtlenR R"tloopt add_callbacktscale_uptworkers_to_closetloggertdebugtretire_workerst scale_down(RR!tto_close((s9lib/python2.7/site-packages/distributed/deploy/cluster.pytscaleSs   cCsyt|jjƒ}td„|jjjƒDƒƒ}td„|jjjƒDƒƒ}t|ƒ}d|||f}|S(Ncss|]}|jVqdS(N(tncores(t.0tws((s9lib/python2.7/site-packages/distributed/deploy/cluster.pys sscss|]}|jVqdS(N(t memory_limit(R/R0((s9lib/python2.7/site-packages/distributed/deploy/cluster.pys tss¢
Workers %d
Cores %d
Memory %s
(R#R R"tsumtvaluesR(RR"tcorestmemoryttext((s9lib/python2.7/site-packages/distributed/deploy/cluster.pyt_widget_statusqs""  cs~y ˆjSWntk rnXddlm}m}m}m}m}m}m }|ddƒ}dˆj j krˆj } d| | f} nd} dt ˆƒj} || ƒ} || ƒ} |ˆjƒd |d dƒƒ‰|d d d d |ƒ‰|d dd |ƒ} |d d dd |ƒ‰|d d dd |ƒ‰|d dd |ƒ} ||ˆ| gƒ|ˆˆ| gƒgd |d dƒƒ}d|_|jd dƒ|jddƒ|| |ˆ|gƒ| gƒ}|ˆ_‡‡‡fd†}| j|ƒ‡‡fd†}| j|ƒtˆj ƒ}‡‡fd†}t|ddˆj jƒ}|ˆj jd<|jƒ|S(s5 Create IPython widget for display within a notebook iÿÿÿÿ(tLayouttVBoxtHBoxtIntTexttButtontHTMLt Accordiontwidtht150pxRs=

Dashboard: %s

ts

%s

tlayoutt min_widthit descriptiontWorkerstScaletMinimumtMaximumtAdaptt500pxsManual ScalingisAdaptive Scalingcs ˆjdˆjdˆjƒdS(Ntminimumtmaximum(Rtvalue(tb(RLRKR(s9lib/python2.7/site-packages/distributed/deploy/cluster.pytadapt_cb¿sc sJtƒ;ˆj}ttƒˆjjƒWdQXˆj|ƒWdQXdS(N(RRMRRR R R-(RNR!(trequestR(s9lib/python2.7/site-packages/distributed/deploy/cluster.pytscale_cbÄs    csˆjƒˆ_dS(N(R7RM((Rtstatus(s9lib/python2.7/site-packages/distributed/deploy/cluster.pyR Ïsiôtio_loops cluster-reprN(t_cached_widgetRt ipywidgetsR8R9R:R;R<R=R>R RR ttypet__name__R7tNonetselected_indext set_titleton_clickRRR$tperiodic_callbackststart(RR8R9R:R;R<R=R>RBtlinkttitlet dashboardR-Rt accordiontboxRORQt scheduler_refR tpc((RLRKRPRRRs9lib/python2.7/site-packages/distributed/deploy/cluster.pyt_widget’sN  4    !' !    cKs|jƒj|S(N(Ret_ipython_display_(RR((s9lib/python2.7/site-packages/distributed/deploy/cluster.pyRfØs( RWt __module__t__doc__RtpropertyRR R-R7ReRf(((s9lib/python2.7/site-packages/distributed/deploy/cluster.pyRs'   ! F(tloggingRtweakrefRRtadaptiveRtutilsRRRRt getLoggerRWR(tobjectR(((s9lib/python2.7/site-packages/distributed/deploy/cluster.pyts   "