MINI MINI MANI MO
ó
Fì ]c @ sT d g Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m Z m Z m Z d d l
m Z m Z d Z
d Z d Z e j ƒ Z d „ Z d e f d
„ ƒ YZ e f e d „ Z d e f d „ ƒ YZ d
e f d „ ƒ YZ e Z d e f d „ ƒ YZ d e f d „ ƒ YZ d e f d „ ƒ YZ d e f d „ ƒ YZ d S( t PooliÿÿÿÿN( t Processt cpu_countt TimeoutError( t Finalizet debugi i i c C s
t | Œ S( N( t map( t args( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt mapstar@ s t MaybeEncodingErrorc B s) e Z d Z d „ Z d „ Z d „ Z RS( sV Wraps possible unpickleable errors, so they can be
safely sent through the socket.c C sA t | ƒ | _ t | ƒ | _ t t | ƒ j | j | j ƒ d S( N( t reprt exct valuet superR t __init__( t selfR R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR K s c C s d | j | j f S( Ns( Error sending result: '%s'. Reason: '%s'( R R ( R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt __str__P s c C s d t | ƒ S( Ns <MaybeEncodingError: %s>( t str( R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt __repr__T s ( t __name__t
__module__t __doc__R R R ( ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR G s c C s¡ | j } | j } t | d ƒ r> | j j ƒ | j j ƒ n | d k rW | | Œ n d } x/| d k s~ | rŽ| | k rŽy
| ƒ } Wn" t t f k
r¯ t d ƒ Pn X| d k rÊ t d ƒ Pn | \ } }
} } }
y t
| | |
Ž f } Wn t k
r} t | f } n Xy | | |
| f ƒ WnM t k
r€} t
| | d ƒ } t d | ƒ | | |
t | f f ƒ n X| d 7} q` Wt d | ƒ d S( Nt _writeri s) worker got EOFError or IOError -- exitings worker got sentinel -- exitingi s0 Possible encoding error while sending result: %ss worker exiting after %d tasks( t putt gett hasattrR t closet _readert Nonet EOFErrort IOErrorR t Truet Exceptiont FalseR ( t inqueuet outqueuet initializert initargst maxtasksR R t completedt taskt jobt it funcR t kwdst resultt et wrapped( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt workerX s>
!
c B s e Z d Z e Z d d d d d „ Z d „ Z d „ Z d „ Z d „ Z d i d „ Z
d d „ Z d d „ Z d d
„ Z
d i d d „ Z d d d „ Z e d
„ ƒ Z e d „ ƒ Z e d „ ƒ Z e d „ ƒ Z d „ Z d „ Z d „ Z d „ Z e d „ ƒ Z e d „ ƒ Z RS( sH
Class which supports an async version of the `apply()` builtin
c
C s) | j ƒ t j ƒ | _ i | _ t | _ | | _ | | _ | | _ | d k r| y
t
ƒ } Wq| t k
rx d } q| Xn | d k r— t d ƒ ‚ n | d k r t
| d ƒ r t d ƒ ‚ n | | _ g | _ | j ƒ t j d t j d | f ƒ | _ t | j _ t | j _ | j j ƒ t j d t j d | j | j | j | j f ƒ | _ t | j _ t | j _ | j j ƒ t j d t j d | j | j | j f ƒ | _ t | j _ t | j _ | j j ƒ t! | | j" d | j | j# | j | j | j | j | j | j f d d ƒ| _$ d S( Ni s& Number of processes must be at least 1t __call__s initializer must be a callablet targetR t exitpriorityi (% t
_setup_queuest Queuet
_taskqueuet _cachet RUNt _statet _maxtasksperchildt _initializert _initargsR R t NotImplementedErrort
ValueErrorR t TypeErrort
_processest _poolt _repopulate_poolt threadingt ThreadR t _handle_workerst _worker_handlerR t daemont startt
_handle_taskst
_quick_putt _outqueuet
_task_handlert _handle_resultst
_quick_gett _result_handlerR t _terminate_poolt _inqueuet
_terminate( R t processesR$ R% t maxtasksperchild( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR ˆ sV
$
c C sw t } xj t t t | j ƒ ƒ ƒ D]M } | j | } | j d k r" t d | ƒ | j ƒ t } | j | =q" q" W| S( s Cleanup after any worker processes which have exited due to reaching
their specified lifetime. Returns True if any workers were cleaned up.
s cleaning up worker %dN(
R! t reversedt ranget lenRA t exitcodeR R t joinR ( R t cleanedR* R0 ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt _join_exited_workers s "
c
C s¦ xŸ t | j t | j ƒ ƒ D] } | j d t d | j | j | j | j | j
f ƒ } | j j | ƒ | j j
d d ƒ | _ t | _ | j ƒ t d ƒ q Wd S( s€ Bring the number of pool processes up to the specified number,
for use after reaping workers which have exited.
R2 R R t
PoolWorkers added workerN( RV R@ RW RA R R0 RQ RK R; R< R: t appendt namet replaceR RG RH R ( R R* t w( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRB Ñ s #
c C s | j ƒ r | j ƒ n d S( sE Clean up any exited workers and start replacements for them.
N( R[ RB ( R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt _maintain_poolá s c C sP d d l m } | ƒ | _ | ƒ | _ | j j j | _ | j j j | _ d S( Ni ( t SimpleQueue(
t queuesRb RQ RK R t sendRJ R t recvRN ( R Rb ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR4 ç s
c C s | j | | | ƒ j ƒ S( s1
Equivalent of `apply()` builtin
( t apply_asyncR ( R R+ R R, ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt applyî s c C s | j | | | ƒ j ƒ S( s/
Equivalent of `map()` builtin
( t map_asyncR ( R R+ t iterablet chunksize( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR õ s i c s¶ | d k rQ t | j ƒ ‰ | j j ‡ ‡ f d † t | ƒ Dƒ ˆ j f ƒ ˆ St j ˆ | | ƒ } t | j ƒ ‰ | j j ‡ f d † t | ƒ Dƒ ˆ j f ƒ d „ ˆ Dƒ Sd S( sZ
Equivalent of `itertools.imap()` -- can be MUCH slower than `Pool.map()`
i c 3 s0 | ]& \ } } ˆ j | ˆ | f i f Vq d S( N( t _job( t .0R* t x( R+ R- ( s, /usr/lib64/python2.7/multiprocessing/pool.pys <genexpr> s c 3 s0 | ]& \ } } ˆ j | t | f i f Vq d S( N( Rk R ( Rl R* Rm ( R- ( s, /usr/lib64/python2.7/multiprocessing/pool.pys <genexpr>
s c s s" | ] } | D] } | Vq
q d S( N( ( Rl t chunkt item( ( s, /usr/lib64/python2.7/multiprocessing/pool.pys <genexpr> s N( t IMapIteratorR7 R6 R t enumeratet _set_lengthR t
_get_tasks( R R+ Ri Rj t task_batches( ( R+ R- s, /usr/lib64/python2.7/multiprocessing/pool.pyt imapü s c s¶ | d k rQ t | j ƒ ‰ | j j ‡ ‡ f d † t | ƒ Dƒ ˆ j f ƒ ˆ St j ˆ | | ƒ } t | j ƒ ‰ | j j ‡ f d † t | ƒ Dƒ ˆ j f ƒ d „ ˆ Dƒ Sd S( sK
Like `imap()` method but ordering of results is arbitrary
i c 3 s0 | ]& \ } } ˆ j | ˆ | f i f Vq d S( N( Rk ( Rl R* Rm ( R+ R- ( s, /usr/lib64/python2.7/multiprocessing/pool.pys <genexpr> s c 3 s0 | ]& \ } } ˆ j | t | f i f Vq d S( N( Rk R ( Rl R* Rm ( R- ( s, /usr/lib64/python2.7/multiprocessing/pool.pys <genexpr> s c s s" | ] } | D] } | Vq
q d S( N( ( Rl Rn Ro ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pys <genexpr> s N( t IMapUnorderedIteratorR7 R6 R Rq Rr R Rs ( R R+ Ri Rj Rt ( ( R+ R- s, /usr/lib64/python2.7/multiprocessing/pool.pyt imap_unordered s c C sA t | j | ƒ } | j j | j d | | | f g d f ƒ | S( s>
Asynchronous equivalent of `apply()` builtin
N( t ApplyResultR7 R6 R Rk R ( R R+ R R, t callbackR- ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRf s +c sæ t | d ƒ s t | ƒ } n | d k rh t t | ƒ t | j ƒ d ƒ \ } } | rh | d 7} qh n t | ƒ d k rƒ d } n t j | | | ƒ } t | j | t | ƒ | ƒ ‰ | j
j ‡ f d † t | ƒ Dƒ d f ƒ ˆ S( s<
Asynchronous equivalent of `map()` builtin
t __len__i i i c 3 s0 | ]& \ } } ˆ j | t | f i f Vq d S( N( Rk R ( Rl R* Rm ( R- ( s, /usr/lib64/python2.7/multiprocessing/pool.pys <genexpr>: s N(
R t listR t divmodRW RA R Rs t MapResultR7 R6 R Rq ( R R+ Ri Rj Ry t extraRt ( ( R- s, /usr/lib64/python2.7/multiprocessing/pool.pyRh ) s ( c C so t j ƒ } xB | j t k s6 | j rP | j t k rP | j ƒ t j d ƒ q W| j j
d ƒ t d ƒ d S( Ngš™™™™™¹?s worker handler exiting(
RC t current_threadR9 R8 R7 t TERMINATERa t timet sleepR6 R R R ( t poolt thread( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRE > s *
c
C s1 t j ƒ } x¶ t | j d ƒ D]˜ \ } } d } x‚ t | ƒ D]P \ } } | j ra t d ƒ Pn y | | ƒ Wq> t k
r t d ƒ Pq> Xq> W| r t d ƒ | | d ƒ q q Pq Wt d ƒ y@ t d ƒ | j d ƒ t d ƒ x | D] } | d ƒ qð WWn t k
r"t d ƒ n Xt d
ƒ d S( Niÿÿÿÿs' task handler found thread._state != RUNs could not put task on queues doing set_length()i s task handler got sentinels/ task handler sending sentinel to result handlers( task handler sending sentinel to workerss/ task handler got IOError when sending sentinelss task handler exiting(
RC R t iterR R Rq R9 R R R (
t taskqueueR R# Rƒ R„ t taskseqt
set_lengthR* R( t p( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRI K s6
c C sÝ t j ƒ } x¡ y
| ƒ } Wn" t t f k
r@ t d ƒ d SX| j rX t d ƒ Pn | d k rr t d ƒ Pn | \ } } } y | | j | | ƒ Wq t k
r¬ q Xq x¢ | rT| j t k rTy
| ƒ } Wn" t t f k
rù t d ƒ d SX| d k rt d ƒ q³ n | \ } } } y | | j | | ƒ Wq³ t k
rPq³ Xq³ Wt
| d ƒ rÀt d ƒ y5 x. t d ƒ D] } | j j
ƒ s—Pn | ƒ q~WWqÀt t f k
r¼qÀXn t d t | ƒ | j ƒ d S( Ns. result handler got EOFError/IOError -- exitings, result handler found thread._state=TERMINATEs result handler got sentinels& result handler ignoring extra sentinelR s" ensuring that outqueue is not fulli
s7 result handler exiting: len(cache)=%s, thread._state=%s( RC R R R R R9 R t _sett KeyErrorR€ R RV R t pollRW ( R# R t cacheR„ R( R) R* t obj( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRM r sV
c c sC t | ƒ } x0 t t j | | ƒ ƒ } | s1 d S| | f Vq d S( N( R… t tuplet itertoolst islice( R+ t itt sizeRm ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRs ¬ s c C s t d ƒ ‚ d S( Ns: pool objects cannot be passed between processes or pickled( R= ( R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt
__reduce__µ s c C s5 t d ƒ | j t k r1 t | _ t | j _ n d S( Ns closing pool( R R9 R8 t CLOSERF ( R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR º s
c C s- t d ƒ t | _ t | j _ | j ƒ d S( Ns terminating pool( R R€ R9 RF RR ( R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt terminateÀ s
c C sS t d ƒ | j j ƒ | j j ƒ | j j ƒ x | j D] } | j ƒ q; Wd S( Ns joining pool( R RF RY RL RO RA ( R R‰ ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRY Æ s
c C sW t d ƒ | j j ƒ x9 | j ƒ rR | j j ƒ rR | j j ƒ t j d ƒ q Wd S( Ns7 removing tasks from inqueue until task handler finishedi ( R t _rlockt acquiret is_aliveR RŒ Re R R‚ ( R" t task_handlerR“ ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt _help_stuff_finishÏ s
c
C s‘ t d ƒ t | _ t | _ t d ƒ | j | | t | ƒ ƒ t | _ | j d ƒ t d ƒ t j ƒ | k r | j d ƒ n | rÔ t
| d d ƒ rÔ t d ƒ x- | D]" } | j d k r« | j ƒ q« q« Wn t d ƒ t j ƒ | k r | j d ƒ n t d ƒ t j ƒ | k r,| j d ƒ n | rt
| d d ƒ rt d
ƒ x; | D]0 } | j
ƒ rVt d | j ƒ | j ƒ qVqVWn d S( Ns finalizing pools& helping task handler/workers to finishs joining worker handlerg}Ô%I²Ti R– s terminating workerss joining task handlers joining result handlers joining pool workerss cleaning up worker %d( R R€ R9 R› RW R R RC R RY R RX R– R™ t pid(
t clsR† R" R# Rƒ t worker_handlerRš t result_handlerR R‰ ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRP Ø s6
N( ( ( ( R R R R R R R[ RB Ra R4 Rg R Ru Rw Rf Rh t staticmethodRE RI RM Rs R” R R– RY R› t classmethodRP ( ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR ‚ s0 9
': Rx c B sD e Z d „ Z d „ Z d „ Z d d „ Z d d „ Z d „ Z RS( c C sS t j t j ƒ ƒ | _ t j ƒ | _ | | _ t | _ | | _
| | | j <d S( N( RC t Conditiont Lockt _condt job_countert nextRk R7 R! t _readyt _callback( R R Ry ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR s c C s | j S( N( R§ ( R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt ready s c C s | j S( N( t _success( R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt
successful s c C sB | j j ƒ z | j s, | j j | ƒ n Wd | j j ƒ Xd S( N( R¤ R˜ R§ t waitt release( R t timeout( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR¬ s
c C s< | j | ƒ | j s t ‚ n | j r/ | j S| j ‚ d S( N( R¬ R§ R Rª t _value( R R® ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR # s
c C s€ | \ | _ | _ | j r7 | j r7 | j | j ƒ n | j j ƒ z t | _ | j j ƒ Wd | j j ƒ X| j | j
=d S( N( Rª R¯ R¨ R¤ R˜ R R§ t notifyR R7 Rk ( R R* RŽ ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRŠ , s
N( R R R R© R« R R¬ R RŠ ( ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRx
s R} c B s e Z d „ Z d „ Z RS( c C s t j | | | ƒ t | _ d g | | _ | | _ | d k r` d | _ t | _ | | j =n | | t
| | ƒ | _ d S( Ni ( Rx R R Rª R R¯ t
_chunksizet _number_leftR§ Rk t bool( R R Rj t lengthRy ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR @ s
c C s | \ } } | r¹ | | j | | j | d | j +| j d 8_ | j d k r| j rn | j | j ƒ n | j | j =| j j ƒ z t | _ | j j
ƒ Wd | j j ƒ XqnW t | _
| | _ | j | j =| j j ƒ z t | _ | j j
ƒ Wd | j j ƒ Xd S( Ni i ( R¯ R± R² R¨ R7 Rk R¤ R˜ R R§ R° R R! Rª ( R R* t success_resultt successR- ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRŠ L s* "
( R R R RŠ ( ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR} > s Rp c B s> e Z d „ Z d „ Z d d „ Z e Z d „ Z d „ Z RS( c C sk t j t j ƒ ƒ | _ t j ƒ | _ | | _ t j ƒ | _
d | _ d | _
i | _ | | | j <d S( Ni ( RC R¢ R£ R¤ R¥ R¦ Rk R7 t collectionst dequet _itemst _indexR t _lengtht _unsorted( R R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR m s c C s | S( N( ( R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt __iter__w s c C sÜ | j j ƒ zž y | j j ƒ } Wn„ t k
r© | j | j k rN t ‚ n | j j | ƒ y | j j ƒ } Wqª t k
r¥ | j | j k rœ t ‚ n t ‚ qª Xn XWd | j j
ƒ X| \ } } | rÒ | S| ‚ d S( N( R¤ R˜ R¹ t popleftt
IndexErrorRº R» t
StopIterationR¬ R R ( R R® Ro R¶ R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR¦ z s&
c C sà | j j ƒ z¾ | j | k r› | j j | ƒ | j d 7_ xJ | j | j k rŠ | j j | j ƒ } | j j | ƒ | j d 7_ qA W| j j ƒ n
| | j | <| j | j k rÊ | j | j
=n Wd | j j ƒ Xd S( Ni ( R¤ R˜ Rº R¹ R] R¼ t popR° R» R7 Rk R ( R R* RŽ ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRŠ “ s
c C s^ | j j ƒ z< | | _ | j | j k rH | j j ƒ | j | j =n Wd | j j ƒ Xd S( N( R¤ R˜ R» Rº R° R7 Rk R ( R R´ ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRr ¦ s
N( R R R R½ R R¦ t __next__RŠ Rr ( ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRp k s
Rv c B s e Z d „ Z RS( c C st | j j ƒ zR | j j | ƒ | j d 7_ | j j ƒ | j | j k r^ | j | j =n Wd | j j ƒ Xd S( Ni (
R¤ R˜ R¹ R] Rº R° R» R7 Rk R ( R R* RŽ ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRŠ ¶ s
( R R RŠ ( ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRv ´ s t
ThreadPoolc B sB e Z d d l m Z d d d d „ Z d „ Z e d „ ƒ Z RS( i ( R c C s t j | | | | ƒ d S( N( R R ( R RS R$ R% ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR É s c C s@ t j ƒ | _ t j ƒ | _ | j j | _ | j j | _ d S( N( R5 RQ RK R RJ R RN ( R ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR4 Ì s c C sW | j j ƒ z5 | j j ƒ | j j d g | ƒ | j j ƒ Wd | j j ƒ Xd S( N( t not_emptyR˜ t queuet cleart extendR t
notify_allR ( R" Rš R“ ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyR› Ò s
N( ( R R t dummyR R R R4 R R› ( ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyRÃ Å s ( t __all__RC R5 R R· R t multiprocessingR R R t multiprocessing.utilR R R8 R• R€ t countR¥ R R R R R0 t objectR Rx t AsyncResultR} Rp Rv RÃ ( ( ( s, /usr/lib64/python2.7/multiprocessing/pool.pyt <module># s, *ÿ ‰.-I
OHA YOOOO