B \2@sDdZddlZddlZddlZddlZddlZddlZddlZddlm Z ddl m Z m Z m Z ddlmZddlmZddlmZddlmZdd lmZddlZdd lmZmZmZmZejrdd lmZejZe d d dZ!dd ddZ"ee e fd ddZ#da$dee e e dddZ%ee d ddZ&Gddde'Z(dS)zUtilities for working with multiple processes, including both forking the server into multiple processes and managing subprocesses. N)hexlify)Future"future_set_result_unless_cancelled%future_set_exception_unless_cancelled)ioloop) PipeIOStream)gen_log)set_close_exec)errno_from_exception)TupleOptionalAnyCallable)List)returnc Cs\tdkr dSytStk r(YnXy tdSttfk rLYnXtddS)z1Returns the number of processors on this machine.NSC_NPROCESSORS_CONFz1Could not detect number of processors; assuming 1) multiprocessing cpu_countNotImplementedErrorossysconfAttributeError ValueErrorrerrorrr.lib/python3.7/site-packages/tornado/process.pyr3s  rcCsjdtjkrdSddl}ytttdd}Wn,tk rZttdt A}YnX| |dS)Nrandomri) sysmodulesrintrrurandomrtimegetpidseed)rr%rrr_reseed_randomCs r&cCs$t\}}t|t|||fS)N)rpiper )rwrrr _pipe_cloexecRs r*) num_processes max_restartsrc s~|dkr d}tdkst|dks(|dkr.t}td|itttdfdd }x$t|D]}||}|dk r`|Sq`Wd}xrnyt \}}Wn4t k r}zt |t j krwWdd}~XYnX|krq|}t|rtd||t|n8t|dkr.td ||t|ntd ||q|d 7}||krXtd ||} | dk r| SqWtddS) aStarts multiple worker processes. If ``num_processes`` is None or <= 0, we detect the number of cores available on this machine and fork that number of child processes. If ``num_processes`` is given and > 0, we fork that specific number of sub-processes. Since we use processes and not threads, there is no shared memory between any server code. Note that multiple processes are not compatible with the autoreload module (or the ``autoreload=True`` option to `tornado.web.Application` which defaults to True when ``debug=True``). When using multiple processes, no IOLoops can be created or referenced until after the call to ``fork_processes``. In each child process, ``fork_processes`` returns its *task id*, a number between 0 and ``num_processes``. Processes that exit abnormally (due to a signal or non-zero exit status) are restarted with the same id (up to ``max_restarts`` times). In the parent process, ``fork_processes`` returns None if all child processes have exited normally, but will otherwise only exit by throwing an exception. max_restarts defaults to 100. NdrzStarting %d processes)ircs.t}|dkrt|a|S||<dSdS)Nr)rforkr&_task_id)r.pid)childrenrr start_childsz#fork_processes..start_childz1child %d (pid %d) killed by signal %d, restartingz3child %d (pid %d) exited with status %d, restartingz!child %d (pid %d) exited normallyrz"Too many child restarts, giving up)r0AssertionErrorrrinfor!r rangerwaitOSErrorr errnoZEINTRpop WIFSIGNALEDZwarningWTERMSIG WEXITSTATUS RuntimeErrorrexit) r+r,r3r.idZ num_restartsr1statuseZnew_idr)r2rfork_processes\sX      rCcCstS)zpReturns the current task id, if any. Returns None if this process was not created by `fork_processes`. )r0rrrrtask_idsrDc@seZdZdZeZdZiZdZe e ddddZ e e gdfdddd Z ded d d dZeddddZeddddZeddddZee ddddZe ddddZdS) Subprocessa Wraps ``subprocess.Popen`` with IOStream support. The constructor is the same as ``subprocess.Popen`` with the following additions: * ``stdin``, ``stdout``, and ``stderr`` may have the value ``tornado.process.Subprocess.STREAM``, which will make the corresponding attribute of the resulting Subprocess a `.PipeIOStream`. If this option is used, the caller is responsible for closing the streams when done with them. The ``Subprocess.STREAM`` option and the ``set_exit_callback`` and ``wait_for_exit`` methods do not work on Windows. There is therefore no reason to use this class instead of ``subprocess.Popen`` on that platform. .. versionchanged:: 5.0 The ``io_loop`` argument (deprecated since version 4.1) has been removed. FN)argskwargsrc Ostj|_g}g}|dtjkrXt\}}||d<|||f| |t ||_ |dtjkrt\}}||d<|||f| |t ||_ |dtjkrt\} } | |d<|| | f| | t | |_ ytj|||_Wn(x|D]} t| qWYnXx|D]} t| q"W|jj|_x.dD]&} t|| sFt|| t|j| qFWd|_d|_dS)Nstdinstdoutstderr)rHrIrJ)rIOLoopcurrentio_loopgetrESTREAMr*extendappendrrHrIrJ subprocessPopenprocrcloser1hasattrsetattrgetattr_exit_callback returncode) selfrFrGZpipe_fdsZto_closeZin_rZin_wZout_rZout_wZerr_rZerr_wfdattrrrr__init__sF               zSubprocess.__init__)callbackrcCs*||_t|tj|j<t|jdS)aRuns ``callback`` when this process exits. The callback takes one argument, the return code of the process. This method uses a ``SIGCHLD`` handler, which is a global setting and may conflict if you have other libraries trying to handle the same signal. If you are using more than one ``IOLoop`` it may be necessary to call `Subprocess.initialize` first to designate one ``IOLoop`` to run the signal handlers. In many cases a close callback on the stdout or stderr streams can be used as an alternative to an exit callback if the signal handler is causing a problem. N)rYrE initialize_waitingr1_try_cleanup_process)r[r_rrrset_exit_callbacks zSubprocess.set_exit_callbackTz Future[int]) raise_errorrcs*ttddfdd }||S)aReturns a `.Future` which resolves when the process exits. Usage:: ret = yield proc.wait_for_exit() This is a coroutine-friendly alternative to `set_exit_callback` (and a replacement for the blocking `subprocess.Popen.wait`). By default, raises `subprocess.CalledProcessError` if the process has a non-zero exit status. Use ``wait_for_exit(raise_error=False)`` to suppress this behavior and return the exit status without raising. .. versionadded:: 4.2 N)retrcs,|dkrrtt|dn t|dS)Nrunknown)rCalledProcessErrorr)re)futurerdrrr_,s z*Subprocess.wait_for_exit..callback)rr!rc)r[rdr_r)rhrdr wait_for_exits zSubprocess.wait_for_exit)rcs8jr dStjttjfdd_d_dS)aInitializes the ``SIGCHLD`` handler. The signal handler is run on an `.IOLoop` to avoid locking issues. Note that the `.IOLoop` used for signal handling need not be the same one used by individual Subprocess objects (as long as the ``IOLoops`` are each running in separate threads). .. versionchanged:: 5.0 The ``io_loop`` argument (deprecated since version 4.1) has been removed. Ncs jS)N)add_callback_from_signal_cleanup)Zsigframe)clsrMrrJsz'Subprocess.initialize..T) _initializedrrKrLsignalSIGCHLD _old_sigchld)rmr)rmrMrr`8s  zSubprocess.initializecCs$|js dSttj|jd|_dS)z Removes the ``SIGCHLD`` handler.NF)rorprqrr)rmrrr uninitializeNszSubprocess.uninitializecCs&x t|jD]}||qWdS)N)listrakeysrb)rmr1rrrrkVszSubprocess._cleanup)r1rc Csyt|tj\}}Wn4tk rJ}zt|tjkr:dSWdd}~XYnX|dkrXdS||ksdt|j |}|j |j |dS)Nr) rwaitpidWNOHANGr8r r9ZECHILDr4rar:rMrj_set_returncode)rmr1Zret_pidrArBZsubprocrrrrb[s  zSubprocess._try_cleanup_process)rArcCs^t|rt| |_nt|s(tt||_|j|j_|jrZ|j}d|_||jdS)N) rr;r<rZ WIFEXITEDr4r=rTrY)r[rAr_rrrrxhs   zSubprocess._set_returncode)T)__name__ __module__ __qualname____doc__objectrOrorarrr r^rr!rcboolri classmethodr`rsrkrbrxrrrrrEs"' rE)N))r}r9rrrprRrr#ZbinasciirZtornado.concurrentrrrZtornadorZtornado.iostreamrZ tornado.logrZtornado.platform.autor Z tornado.utilr typingr r r rZ TYPE_CHECKINGrrgr!rr&r*r0rCrDr~rErrrrs4       ^