U ,0[@sdZddlmZmZmZddlZddlZddlZddlZddl m Z m Z ddl m Z ddlmZddlmZmZddlmZmZdd lmZdd lmZmZdd lmZeZGd d d eZGdddeZ GdddeZ!ddZ"ddZ#dcddZ$ddZ%ddZ&eedddddd d!d"Z'd#d$Z(d%d&Z)d'd(Z*e+Z,e+Z-eGd)d*d*Z.d+d,Z/d-d.Z0eGd/d0d0Z1eGd1d2d2eZ2Gd3d4d4e.Z3ddd6d7Z4ded8d9Z5d:Z6d5Z7eGd;d<d<Z8d=d>Z9eedd?ddd@dAdBZ:GdCdDdDe;ZGdGdHdHe+Z?ej@dIdJZAdKdLZBGdMdNdNeZCdOdPZDGdQdRdRe+ZEGdSdTdTeEZFGdUdVdVeEZGGdWdXdXeZHGdYdZdZeZIGd[d\d\e+ZJGd]d^d^eZKGd_d`d`ejLZMd*d4dddadbd dd9dd.dappendcalled _runCallbacks)rFr!r& callbackArgscallbackKeywords errbackArgserrbackKeywordsZcbsrrr addCallbacks%s   zDeferred.addCallbackscOs|j|||dS)z^ Convenience method for adding just a callback. See L{addCallbacks}. )rMrNrQrFr!r)r*rrr addCallback;szDeferred.addCallbackcOs|jt|||dS)z^ Convenience method for adding just an errback. See L{addCallbacks}. )rOrP)rQr9)rFr&r)r*rrr addErrbackEszDeferred.addErrbackcOs|j||||||dS)z Convenience method for adding a single callable as both a callback and an errback. See L{addCallbacks}. )rMrOrNrPrRrSrrraddBothPs zDeferred.addBothcsTdgfdd}||fdd}|fdd}|S)a Time out this L{Deferred} by scheduling it to be cancelled after C{timeout} seconds. The timeout encompasses all the callbacks and errbacks added to this L{defer.Deferred} before the call to L{addTimeout}, and none added after the call. If this L{Deferred} gets timed out, it errbacks with a L{TimeoutError}, unless a cancelable function was passed to its initialization or unless a different C{onTimeoutCancel} callable is provided. @param timeout: number of seconds to wait before timing out this L{Deferred} @type timeout: L{int} @param clock: The object which will be used to schedule the timeout. @type clock: L{twisted.internet.interfaces.IReactorTime} @param onTimeoutCancel: A callable which is called immediately after this L{Deferred} times out, and not if this L{Deferred} is otherwise cancelled before the timeout. It takes an arbitrary value, which is the value of this L{Deferred} at that exact point in time (probably a L{CancelledError} L{Failure}), and the C{timeout}. The default callable (if none is provided) will translate a L{CancelledError} L{Failure} into a L{TimeoutError}. @type onTimeoutCancel: L{callable} @return: C{self}. @rtype: a L{Deferred} @since: 16.5 Fcsdd<dS)NTr)cancelr)rFtimedOutrr timeItOutsz&Deferred.addTimeout..timeItOutcsdrpt}||S|S)Nr)_cancelledToTimedOutError)valueZtoCall)onTimeoutCancelrXr6rrconvertCancelleds z-Deferred.addTimeout..convertCancelledcsr|Sr7)activerWr#) delayedCallrr cancelTimeoutsz*Deferred.addTimeout..cancelTimeout) callLaterrV)rFr6Zclockr\rYr]rar)r`r\rFrXr6r addTimeout\s"    zDeferred.addTimeoutcCs||_||j|jS)a Chain another L{Deferred} to this L{Deferred}. This method adds callbacks to this L{Deferred} to call C{d}'s callback or errback, as appropriate. It is merely a shorthand way of performing the following:: self.addCallbacks(d.callback, d.errback) When you chain a deferred d2 to another deferred d1 with d1.chainDeferred(d2), you are making d2 participate in the callback chain of d1. Thus any event that fires d1 will also fire d2. However, the converse is B{not} true; if d2 is fired d1 will not be affected. Note that unlike the case where chaining is caused by a L{Deferred} being returned from a callback, it is possible to cause the call stack size limit to be exceeded by chaining many L{Deferred}s together with C{chainDeferred}. @return: C{self}. @rtype: a L{Deferred} ) _chainedTorQr!r&rFr$rrr chainDeferredszDeferred.chainDeferredcCst|trt||dS)a Run all success callbacks that have been added to this L{Deferred}. Each callback will have its result passed as the first argument to the next; this way, the callbacks act as a 'processing chain'. If the success-callback returns a L{Failure} or raises an L{Exception}, processing will continue on the *error* callback chain. If a callback (or errback) returns another L{Deferred}, this L{Deferred} will be chained to it (and further callbacks will not run until that L{Deferred} has a result). An instance of L{Deferred} may only have either L{callback} or L{errback} called on it, and only once. @param result: The object which will be passed to the first callback added to this L{Deferred} (via L{addCallback}). @raise AlreadyCalledError: If L{callback} or L{errback} has already been called on this L{Deferred}. N)r/r rI_startRunCallbacks)rFr#rrrr!szDeferred.callbackcCs<|dkrtj|jd}nt|tjs.t|}||dS)a Run all error callbacks that have been added to this L{Deferred}. Each callback will have its result passed as the first argument to the next; this way, the callbacks act as a 'processing chain'. Also, if the error-callback returns a non-Failure or doesn't raise an L{Exception}, processing will continue on the *success*-callback chain. If the argument that's passed to me is not a L{failure.Failure} instance, it will be embedded in one. If no argument is passed, a L{failure.Failure} instance will be created based on the current traceback stack. Passing a string as `fail' is deprecated, and will be punished with a warning message. An instance of L{Deferred} may only have either L{callback} or L{errback} called on it, and only once. @param fail: The L{Failure} object which will be passed to the first errback added to this L{Deferred} (via L{addErrback}). Alternatively, a L{Exception} instance from which a L{Failure} will be constructed (with no traceback) or L{None} to create a L{Failure} instance from the current exception state (with a traceback). @raise AlreadyCalledError: If L{callback} or L{errback} has already been called on this L{Deferred}. @raise NoCurrentExceptionError: If C{fail} is L{None} but there is no current exception state. Nr,)r r-r.r/rg)rFr'rrrr&s !  zDeferred.errbackcCs|jd|_dS)zP Stop processing on a L{Deferred} until L{unpause}() is called. r3N)pausedrFrrrpauseszDeferred.pausecCs(|jd|_|jrdS|jr$|dS)zI Process all callbacks made since L{pause}() was called. r3N)rhrKrLrirrrunpauses  zDeferred.unpausecCsT|js:|j}|r||nd|_|jsP|ttnt|jt rP|j dS)a Cancel this L{Deferred}. If the L{Deferred} has not yet had its C{errback} or C{callback} method invoked, call the canceller function provided to the constructor. If that function does not invoke C{callback} or C{errback}, or if no canceller function was provided, errback with L{CancelledError}. If this L{Deferred} is waiting on another L{Deferred}, forward the cancellation to the other L{Deferred}. TN) rKr?_suppressAlreadyCalledr&r r-rr/r#r rWrErrrrW s   zDeferred.cancelcCs|jrH|jrd|_dS|jrD|jdkr.t|_d|j}t|t|jrt|jdkr`t|_tdd|j_ d|_||_ | dS)NF T) rKrlr.rAr@_getDebugTracebacksrrBrCinvokerr#rL)rFr#Zextrarrrrg's"  zDeferred._startRunCallbackscCst|fdft|fdffS)zJ Build a tuple of callback and errback with L{_CONTINUE}. N) _CONTINUErirrr _continuation;s  zDeferred._continuationc Cs|jr dS|g}|r|d}|jr(dSd}d|_|jr|jd}|t|jtj\}}}|pdd}|pli}|t kr|d}|j|_d|_|j dk rd|j _ |jd8_| |d}qz@d|_z,||jf|||_|j|krt |dW5d|_XWntj|jd |_Yq2Xt|jtr2t|jd t} | tksVt| tsV|jjr~||j|_|jj |qq2d|j_|jj dk rd|jj _ | |_q2|rt|jtjr|j|j dkrt|_ |j|j _ n|j dk rd|j _ |qdS) a Run the chain of callbacks once a result is available. This consists of a simple loop over all of the callbacks, calling each with the current result and making the current result equal to the return value (or raised exception) of that call. If L{_runningCallbacks} is true, this loop won't run at all, since it is already running above us on the call stack. If C{self.paused} is true, the loop also won't run, because that's what it means to be paused. The loop will terminate before processing all of the callbacks if a L{Deferred} without a result is encountered. If a L{Deferred} I{with} a result is encountered, that result is taken and the loop proceeds. @note: The implementation is complicated slightly by the fact that chaining (associating two L{Deferred}s with each other such that one will wait for the result of the other, as happens when a Deferred is returned from a callback on another L{Deferred}) is supported iteratively rather than recursively, to avoid running out of stack frames when processing long chains. Nr=Trrr3Fz|Callback returned the Deferred it was attached to; this breaks the callback chain and will raise an exception in the future.r,r#)_runningCallbacksrhrdr>popr/r#r r-rqrA failResultrJrr.r getattr _NO_RESULTrjrrZ cleanFailurer@) rFchainZcurrentZfinisheditemr!r)r*ZchaineeZ resultResultrrrrLCst             zDeferred._runCallbackscCs^|jj}t|dt}t|}|jdk r8dt|jf}n|tkrFd}n d|f}d|||fS)zE Return a string representation of this C{Deferred}. r#Nz waiting on Deferred at 0x%xz current result: %rz<%s at 0x%x%s>) __class__rrvrwidrd)rFZcnamer#ZmyIDrrr__str__s   zDeferred.__str__cCs|Sr7rrirrr__iter__szDeferred.__iter__cCsR|jr |St|dt}|tkr"|St|tjrFd|j_||j_ |jnt |dS)Nr#) rhrvrwr/r r-rArur[Z __failure__ StopIteration)rFr[r#rrrsends  z Deferred.sendcs~z j}Wn.tk r8ddlmfdd}YnX|fdd}fdd}fd d }|||S) a Adapt a L{Deferred} into a L{asyncio.Future} which is bound to C{loop}. @note: converting a L{Deferred} to an L{asyncio.Future} consumes both its result and its errors, so this method implicitly converts C{self} into a L{Deferred} firing with L{None}, regardless of what its result previously would have been. @since: Twisted 17.5.0 @param loop: The asyncio event loop to bind the L{asyncio.Future} to. @type loop: L{asyncio.AbstractEventLoop} or similar @param deferred: The Deferred to adapt. @type deferred: L{Deferred} @return: A Future which will fire when the Deferred fires. @rtype: L{asyncio.Future} r)Futurecs dS)N)looprr)rrrr createFuturesz'Deferred.asFuture..createFuturecs|rdSr7) cancelledrW)Z futureAgainrirr checkCancelsz&Deferred.asFuture..checkCancelcss|jdSr7)rZ set_exceptionr[)r futurerr maybeFailsz$Deferred.asFuture..maybeFailcss|dSr7)rZ set_resultr_rrr maybeSucceedsz'Deferred.asFuture..maybeSucceed)Z create_futureAttributeErrorasynciorrQadd_done_callback)rFrrrrrr)rrrrFrasFutures       zDeferred.asFuturecsTfddtfdd}||}|_fdd}|||S)a Adapt an L{asyncio.Future} to a L{Deferred}. @note: This creates a L{Deferred} from a L{asyncio.Future}, I{not} from a C{coroutine}; in other words, you will need to call L{asyncio.ensure_future}, L{asyncio.loop.create_task} or create an L{asyncio.Task} yourself to get from a C{coroutine} to a L{asyncio.Future} if what you have is an awaitable coroutine and not a L{asyncio.Future}. (The length of this list of techniques is exactly why we have left it to the caller!) @since: Twisted 17.5.0 @param future: The Future to adapt. @type future: L{asyncio.Future} @return: A Deferred which will fire when the Future fires. @rtype: L{Deferred} cs2z |}Wnt}YnXj|dSr7)r#r r-actualr!)r#Z extracted)adaptrrr6s  z"Deferred.fromFuture..adaptcs|dSr7)rWr!)Zreself)r futureCancelrrrW=sz#Deferred.fromFuture..cancelcs|krt_jS|Sr7)r rr_)rrrruncancelBsz%Deferred.fromFuture..uncancel)objectrrTr)clsrrWrFrr)rrrr fromFuture s   zDeferred.fromFuture)N)NNNNN)N)N)N)%rrrrrKrhrArlrsr.rdrHrQrTrUrVrcrfr!r&rjrkrWrgrrrLr}__repr__r~r _extraneousr __await____next__r classmethodrrrrrr sN,     > )  )r cCs$t|tjr |tt|d|S)a. A default translation function that translates L{Failure}s that are L{CancelledError}s to L{TimeoutError}s. @param value: Anything @type value: Anything @param timeout: The timeout @type timeout: L{int} @rtype: C{value} @raise: L{TimeoutError} @since: 16.5 r )r/r r-traprr)r[r6rrrrZMs   rZcCsVddlm}tdkr:ddlm}||s2t||r:t|St|tsRtd|f|S)a Schedule the execution of a coroutine that awaits/yields from L{Deferred}s, wrapping it in a L{Deferred} that will fire on success/failure of the coroutine. If a Deferred is passed to this function, it will be returned directly (mimicing C{asyncio}'s C{ensure_future} function). Coroutine functions return a coroutine object, similar to how generators work. This function turns that coroutine into a Deferred, meaning that it can be used in regular Twisted code. For example:: import treq from twisted.internet.defer import ensureDeferred from twisted.internet.task import react async def crawl(pages): results = {} for page in pages: results[page] = await treq.content(await treq.get(page)) return results def main(reactor): pages = [ "http://localhost:8080" ] d = ensureDeferred(crawl(pages)) d.addCallback(print) return d react(main) @param coro: The coroutine object to schedule, or a L{Deferred}. @type coro: A Python 3.5+ C{async def} C{coroutine}, a Python 3.4+ C{yield from} using L{types.GeneratorType}, or a L{Deferred}. @rtype: L{Deferred} r) GeneratorType)r) iscoroutinez#%r is not a coroutine or a Deferred) typesrrrrr/_cancellableInlineCallbacksr ValueError)cororrrrrensureDeferredds%   rc@s$eZdZdZdZddZddZdS)r@z Deferred debug helper. NcCstd}t|dr:|d7}|d|jdd7}|d7}t|drp|d7}|d|jdd7}|d7}|S) NrzrDz C: Deferred was created: C:rmz C:rpz I: First Invoker was: I:z I:)hasattrjoinrDrstripreplacerp)rFinforrrros  zDebugInfo._getDebugTracebackscCsD|jdk r@tjddd|}|r*d}nd}tj||j|ddS)z Print tracebacks and die. If the *last* (and I do mean *last*) callback leaves me in an error state, print a traceback (if said errback is a L{Failure}). NzUnhandled error in Deferred:T)ZisErrorz(debug: {debugInfo})) debugInfo)rurZcriticalror )rFrformatrrr__del__s zDebugInfo.__del__)rrrrrurorrrrrr@s r@c@s0eZdZdZddZddZddZdd Zd S) FirstErrora! First error to occur in a L{DeferredList} if C{fireOnOneErrback} is set. @ivar subFailure: The L{Failure} that occurred. @type subFailure: L{Failure} @ivar index: The index of the L{Deferred} in the L{DeferredList} where it happened. @type index: L{int} cCst|||||_||_dSr7) ExceptionrH subFailureindex)rFr rrrrrHszFirstError.__init__cCsd|j|jjfS)z The I{repr} of L{FirstError} instances includes the repr of the wrapped failure's exception and the index of the L{FirstError}. zFirstError[#%d, %r])rrr[rirrrrszFirstError.__repr__cCsd|j|jfS)z The I{str} of L{FirstError} instances includes the I{str} of the entire wrapped failure (including its traceback and exception) and the index of the L{FirstError}. zFirstError[#%d, %s])rrrirrrr}szFirstError.__str__cCs(t|tr$t|j|jf|j|jfSdS)a+ Comparison between L{FirstError} and other L{FirstError} instances is defined as the comparison of the index and sub-failure of each instance. L{FirstError} instances don't compare equal to anything that isn't a L{FirstError} instance. @since: 8.2 r=)r/rr rr)rFotherrrr__cmp__s   zFirstError.__cmp__N)rrrrrHrr}rrrrrrs   rc@s2eZdZdZdZdZd ddZddZddZd S) DeferredLista) L{DeferredList} is a tool for collecting the results of several Deferreds. This tracks a list of L{Deferred}s for their results, and makes a single callback when they have all completed. By default, the ultimate result is a list of (success, result) tuples, 'success' being a boolean. L{DeferredList} exposes the same API that L{Deferred} does, so callbacks and errbacks can be added to it in the same way. L{DeferredList} is implemented by adding callbacks and errbacks to each L{Deferred} in the list passed to it. This means callbacks and errbacks added to the Deferreds before they are passed to L{DeferredList} will change the result that L{DeferredList} sees (i.e., L{DeferredList} is not special). Callbacks and errbacks can also be added to the Deferreds after they are passed to L{DeferredList} and L{DeferredList} may change the result that they see. See the documentation for the C{__init__} arguments for more information. @ivar _deferredList: The L{list} of L{Deferred}s to track. FcCst||_dgt|j|_t|t|jdkrD|sD||j||_||_||_ d|_ d}|jD]*}|j |j |j |t f|tfd|d}qfdS)a  Initialize a DeferredList. @param deferredList: The list of deferreds to track. @type deferredList: L{list} of L{Deferred}s @param fireOnOneCallback: (keyword param) a flag indicating that this L{DeferredList} will fire when the first L{Deferred} in C{deferredList} fires with a non-failure result without waiting for any of the other Deferreds. When this flag is set, the DeferredList will fire with a two-tuple: the first element is the result of the Deferred which fired; the second element is the index in C{deferredList} of that Deferred. @type fireOnOneCallback: L{bool} @param fireOnOneErrback: (keyword param) a flag indicating that this L{DeferredList} will fire when the first L{Deferred} in C{deferredList} fires with a failure result without waiting for any of the other Deferreds. When this flag is set, if a Deferred in the list errbacks, the DeferredList will errback with a L{FirstError} failure wrapping the failure of that Deferred. @type fireOnOneErrback: L{bool} @param consumeErrors: (keyword param) a flag indicating that failures in any of the included L{Deferred}s should not be propagated to errbacks added to the individual L{Deferred}s after this L{DeferredList} is constructed. After constructing the L{DeferredList}, any errors in the individual L{Deferred}s will be converted to a callback result of L{None}. This is useful to prevent spurious 'Unhandled error in Deferred' messages from being logged. This does not prevent C{fireOnOneErrback} from working. @type consumeErrors: L{bool} Nr)rMrOr3)list _deferredListlen resultListr rHr!fireOnOneCallbackfireOnOneErrback consumeErrors finishedCountrQ _cbDeferredSUCCESSFAILURE)rF deferredListrrrrr5rrrrHs #     zDeferredList.__init__cCs||f|j|<|jd7_|js|tkr@|jr@|||fnB|tkrf|jrf|t t ||n|jt |jkr||j|tkr|j rd}|S)zI (internal) Callback for when one of my deferreds fires. r3N)rrrKrrr!rrr&r r-rrr)rFr#rZ succeededrrrrOs zDeferredList._cbDeferredcCs:|js6|jD](}z |Wq tdYq Xq dS)a Cancel this L{DeferredList}. If the L{DeferredList} hasn't fired yet, cancel every L{Deferred} in the list. If the L{DeferredList} has fired, including the case where the C{fireOnOneCallback}/C{fireOnOneErrback} flag is set and the L{DeferredList} fires because one L{Deferred} in the list fires with a non-failure/failure result, do nothing in the C{cancel} method. z-Exception raised from user supplied cancellerN)rKrrWrr )rFr5rrrrWds   zDeferredList.cancelN)FFF) rrrrrrrHrrWrrrrrs 9rFcCs$|D]\}}|stqdd|DS)NcSsg|] }|dqS)r3r).0xrrr sz%_parseDListResult..)rI)lrZsuccessr[rrr_parseDListResult{s  rcCst|d|d}|t|S)as Returns, via a L{Deferred}, a list with the results of the given L{Deferred}s - in effect, a "join" of multiple deferred operations. The returned L{Deferred} will fire when I{all} of the provided L{Deferred}s have fired, or when any one of them has failed. This method can be cancelled by calling the C{cancel} method of the L{Deferred}, all the L{Deferred}s in the list will be cancelled. This differs from L{DeferredList} in that you don't need to parse the result for success/failure. @type deferredList: L{list} of L{Deferred}s @param consumeErrors: (keyword param) a flag, defaulting to False, indicating that failures in any of the given L{Deferred}s should not be propagated to errbacks added to the individual L{Deferred}s after this L{gatherResults} invocation. Any such errors in the individual L{Deferred}s will be converted to a callback result of L{None}. This is useful to prevent spurious 'Unhandled error in Deferred' messages from being logged. This parameter is available since 11.1.0. @type consumeErrors: L{bool} T)rr)rrTr)rrr$rrr gatherResultss  rTc@s eZdZdZddZddZdS)waitForDeferred# See L{deferredGenerator}. cCs2tjdtddt|ts(td|f||_dS)Nztwisted.internet.defer.waitForDeferred was deprecated in Twisted 15.0.0; please use twisted.internet.defer.inlineCallbacks instead) stacklevelz9You must give waitForDeferred a Deferred. You gave it %r.)warningswarnDeprecationWarningr/r TypeErrorr$rerrrrHs zwaitForDeferred.__init__cCst|jtjr|j|jSr7)r/r#r r-ZraiseExceptionrirrr getResults zwaitForDeferred.getResultN)rrrrrHrrrrrrs rcsd}ddgz t}Wn8tk r:|YSYSXt|trhttdSt|tr |ffdd }|j |drdd<Sdd<dd<d}q dS) rNTz Yield waitForDeferred(d), not d!cs.||_dr dd<|d<n tdSNrFr3)r#_deferGenerator)rr#r5gwaitingrr gotResults  z"_deferGenerator..gotResultrFr3) nextrr!r&r/r r'rrr$rV)rr5r#rrrrrs.       rz&twisted.internet.defer.inlineCallbackscstfdd}|S)aI L{deferredGenerator} and L{waitForDeferred} help you write L{Deferred}-using code that looks like a regular sequential function. Consider the use of L{inlineCallbacks} instead, which can accomplish the same thing in a more concise manner. There are two important functions involved: L{waitForDeferred}, and L{deferredGenerator}. They are used together, like this:: @deferredGenerator def thingummy(): thing = waitForDeferred(makeSomeRequestResultingInDeferred()) yield thing thing = thing.getResult() print(thing) #the result! hoorj! L{waitForDeferred} returns something that you should immediately yield; when your generator is resumed, calling C{thing.getResult()} will either give you the result of the L{Deferred} if it was a success, or raise an exception if it was a failure. Calling C{getResult} is B{absolutely mandatory}. If you do not call it, I{your program will not work}. L{deferredGenerator} takes one of these waitForDeferred-using generator functions and converts it into a function that returns a L{Deferred}. The result of the L{Deferred} will be the last value that your generator yielded unless the last value is a L{waitForDeferred} instance, in which case the result will be L{None}. If the function raises an unhandled exception, the L{Deferred} will errback instead. Remember that C{return result} won't work; use C{yield result; return} in place of that. Note that not yielding anything from your generator will make the L{Deferred} result in L{None}. Yielding a L{Deferred} from your generator is also an error condition; always yield C{waitForDeferred(d)} instead. The L{Deferred} returned from your deferred generator may also errback if your generator raised an exception. For example:: @deferredGenerator def thingummy(): thing = waitForDeferred(makeSomeRequestResultingInDeferred()) yield thing thing = thing.getResult() if thing == 'I love Twisted': # will become the result of the Deferred yield 'TWISTED IS GREAT!' return else: # will trigger an errback raise Exception('DESTROY ALL LIFE') Put succinctly, these functions connect deferred-using code with this 'fake blocking' style in both directions: L{waitForDeferred} converts from a L{Deferred} to the 'blocking' style, and L{deferredGenerator} converts from the 'blocking' style to a L{Deferred}. cst||tSr7)rr )r)kwargsr0rrunwindGenerator5sz*deferredGenerator..unwindGeneratorrr0rrrrdeferredGenerators:rc@seZdZddZdS)_DefGen_ReturncCs ||_dSr7)r[)rFr[rrrrH@sz_DefGen_Return.__init__N)rrrrHrrrrr?srcCs t|dS)a Return val from a L{inlineCallbacks} generator. Note: this is currently implemented by raising an exception derived from L{BaseException}. You might want to change any 'except:' clauses to an 'except Exception:' clause so as not to catch this exception. Also: while this function currently will work when called from within arbitrary functions called from within the generator, do not rely upon this behavior. N)r)valrrr returnValueEs rc@s$eZdZdZeZejddZdS)_CancellationStatusa- Cancellation status of an L{inlineCallbacks} invocation. @ivar waitingOn: the L{Deferred} being waited upon (which L{_inlineCallbacks} must fill out before returning) @ivar deferred: the L{Deferred} to callback or errback when the generator invocation has finished. N)default)rrrrattrZibr5 waitingOnrrrrrVs rc sddgz*t|tj}|r&|}n |}Wntk rn}zjt|ddWYdSd}~XYnt k r }z~t dj }|r|j }|j j r|}|j j r|j }q|j j j}|j}td|j j j|j j jft||j|jWYdSd}~XYnjYdSXt|trfdd} || drbd d<|_dSd }dd<dd <qdS) a/ Carry out the work of L{inlineCallbacks}. Iterate the generator produced by an C{@}L{inlineCallbacks}-decorated function, C{g}, C{send()}ing it the results of each value C{yield}ed by that generator, until a L{Deferred} is yielded, at which point a callback is added to that L{Deferred} to call this function again. @param result: The last result seen by this generator. Note that this is never a L{Deferred} - by the time this function is invoked, the L{Deferred} has been called back and this will be a particular result at a point in its callback chain. @param g: a generator object returned by calling a function or method decorated with C{@}L{inlineCallbacks} @param status: a L{_CancellationStatus} tracking the current status of C{g} TNr[rzvreturnValue() in %r causing %r to exit: returnValue should only be invoked by functions decorated with inlineCallbackscs*drdd<|d<n t|dSr)_inlineCallbacks)rrstatusrrrrs z#_inlineCallbacks..gotResultrFr3)r/r r-ZthrowExceptionIntoGeneratorrrr5r!rvrrtb_nexttb_framef_code co_filename tb_linenor warn_explicitco_namerr[r&r rVr) r#rrZ isFailureeZ appCodeTraceZ ultimateTracefilenamelinenorrrrrgs\        rcs:fddt}t|fddtd||S)z Make an C{@}L{inlineCallbacks} cancellable. @param g: a generator object returned by calling a function or method decorated with C{@}L{inlineCallbacks} @return: L{Deferred} for the C{@}L{inlineCallbacks} that is cancellable. cs4g|j|_}||j||tdSr7)r>rUextendr&&_InternalInlineCallbacksCancelledError)itZtmp) handleCancelrrrWs  z+_cancellableInlineCallbacks..cancelcs(|tt_j}|jS)aR Propagate the cancellation of an C{@}L{inlineCallbacks} to the L{Deferred} it is waiting on. @param result: An L{_InternalInlineCallbacksCancelledError} from C{cancel()}. @return: A new L{Deferred} that the C{@}L{inlineCallback} generator can callback or errback through. )rrr r5rrW)r#Zawaited)rWrrrrs  z1_cancellableInlineCallbacks..handleCancelN)r rr)rr5r)rWrrrrs  rc@seZdZdZdS)rz A unique exception used only in L{_cancellableInlineCallbacks} to verify that an L{inlineCallbacks} is being cancelled as expected. Nrrrrrrsrcstfdd}|S)a L{inlineCallbacks} helps you write L{Deferred}-using code that looks like a regular sequential function. For example:: @inlineCallbacks def thingummy(): thing = yield makeSomeRequestResultingInDeferred() print(thing) # the result! hoorj! When you call anything that results in a L{Deferred}, you can simply yield it; your generator will automatically be resumed when the Deferred's result is available. The generator will be sent the result of the L{Deferred} with the 'send' method on generators, or if the result was a failure, 'throw'. Things that are not L{Deferred}s may also be yielded, and your generator will be resumed with the same object sent back. This means C{yield} performs an operation roughly equivalent to L{maybeDeferred}. Your inlineCallbacks-enabled generator will return a L{Deferred} object, which will result in the return value of the generator (or will fail with a failure object if your generator raises an unhandled exception). Note that you can't use C{return result} to return a value; use C{returnValue(result)} instead. Falling off the end of the generator, or simply using C{return} will cause the L{Deferred} to have a result of L{None}. Be aware that L{returnValue} will not accept a L{Deferred} as a parameter. If you believe the thing you'd like to return could be a L{Deferred}, do this:: result = yield result returnValue(result) The L{Deferred} returned from your deferred generator may errback if your generator raised an exception:: @inlineCallbacks def thingummy(): thing = yield makeSomeRequestResultingInDeferred() if thing == 'I love Twisted': # will become the result of the Deferred returnValue('TWISTED IS GREAT!') else: # will trigger an errback raise Exception('DESTROY ALL LIFE') It is possible to use the C{return} statement instead of L{returnValue}:: @inlineCallbacks def loadData(url): response = yield makeRequest(url) return json.loads(response) You can cancel the L{Deferred} returned from your L{inlineCallbacks} generator before it is fired by your generator completing (either by reaching its end, a C{return} statement, or by calling L{returnValue}). A C{CancelledError} will be raised from the C{yielde}ed L{Deferred} that has been cancelled if that C{Deferred} does not otherwise suppress it. csVz||}Wn"tk r0tdfYnXt|tjsNtd|ft|S)NzkinlineCallbacks requires %r to produce a generator; insteadcaught returnValue being used in a non-generatorzBinlineCallbacks requires %r to produce a generator; instead got %r)rrr/rrr)r)rgenrrrrAs  z(inlineCallbacks..unwindGeneratorrrrrrinlineCallbackss; rc@s$eZdZddZddZddZdS)_ConcurrencyPrimitivecCs g|_dSr7)rrirrrrHTsz_ConcurrencyPrimitive.__init__cCs ||Sr7)release)rFrrrr_releaseAndReturnXsz'_ConcurrencyPrimitive._releaseAndReturncsrtdkr.stdtddjjfdd\ddfdd}}|||S)a Acquire, run, release. This function takes a callable as its first argument and any number of other positional and keyword arguments. When the lock or semaphore is acquired, the callable will be invoked with those arguments. The callable may return a L{Deferred}; if it does, the lock or semaphore won't be released until that L{Deferred} fires. @return: L{Deferred} of function result. rz-run() takes at least 2 arguments, none given.z,%s.run() takes at least 2 arguments, 1 givenrNcs tf}|j|Sr7)r1rVr)Z ignoredResultr$r)r0rrFrrr+ss z*_ConcurrencyPrimitive.run..execute)rrr{racquirerT)r)rr+r$rrrrun]s    z_ConcurrencyPrimitive.runN)rrrrHrrrrrrrSsrc@s,eZdZdZdZddZddZddZd S) DeferredLockz A lock for event driven systems. @ivar locked: C{True} when this Lock has been acquired, false at all other times. Do not change this value, but it is useful to examine for the equivalent of a "non-blocking" acquisition. FcCs|j|dSa Remove a deferred d from our waiting list, as the deferred has been canceled. Note: We do not need to wrap this in a try/except to catch d not being in self.waiting because this canceller will not be called if d has fired. release() pops a deferred out of self.waiting and calls it, so the canceller will no longer be called. @param d: The deferred that has been canceled. Nrremovererrr_cancelAcquires zDeferredLock._cancelAcquirecCs4t|jd}|jr |j|nd|_|||S)aL Attempt to acquire the lock. Returns a L{Deferred} that fires on lock acquisition with the L{DeferredLock} as the value. If the lock is locked, then the Deferred is placed at the end of a waiting list. @return: a L{Deferred} which fires on lock acquisition. @rtype: a L{Deferred} rGT)r rlockedrrJr!rerrrrs  zDeferredLock.acquirecCs:|jstdd|_|jr6d|_|jd}||dS)z Release the lock. If there is a waiting list, then the first L{Deferred} in that waiting list will be called back. Should be called by whomever did the L{acquire}() when the shared resource is free. z!Tried to release an unlocked lockFTrN)rrIrrtr!rerrrrs  zDeferredLock.releaseN)rrrrrrrrrrrrr~s rc@s0eZdZdZddZddZddZdd Zd S) DeferredSemaphorea A semaphore for event driven systems. If you are looking into this as a means of limiting parallelism, you might find L{twisted.internet.task.Cooperator} more useful. @ivar limit: At most this many users may acquire this semaphore at once. @type limit: L{int} @ivar tokens: The difference between C{limit} and the number of users which have currently acquired this semaphore. @type tokens: L{int} cCs*t||dkrtd||_||_dS)ze @param tokens: initial value of L{tokens} and L{limit} @type tokens: L{int} r3z&DeferredSemaphore requires tokens >= 1N)rrHrtokenslimit)rFrrrrrHs  zDeferredSemaphore.__init__cCs|j|dSrrrerrrrs z DeferredSemaphore._cancelAcquirecCsL|jdkstdt|jd}|js2|j|n|jd|_|||S)zq Attempt to acquire the token. @return: a L{Deferred} which fires on token acquisition. rz9Internal inconsistency?? tokens should never be negativerr3)rrIr rrrJr!rerrrrs   zDeferredSemaphore.acquirecCsL|j|jkstd|jd|_|jrH|jd|_|jd}||dS)z Release the token. Should be called by whoever did the L{acquire}() when the shared resource is free. z4Someone released me too many times: too many tokens!r3rN)rrrIrrtr!rerrrrs    zDeferredSemaphore.releaseN)rrrrrHrrrrrrrrs  rc@s eZdZdS) QueueOverflowNrrrrrr src@s eZdZdS)QueueUnderflowNrrrrrrsrc@s2eZdZdZd ddZddZddZd d ZdS) DeferredQueuea An event driven queue. Objects may be added as usual to this queue. When an attempt is made to retrieve an object when the queue is empty, a L{Deferred} is returned which will fire when an object becomes available. @ivar size: The maximum number of objects to allow into the queue at a time. When an attempt to add a new object would exceed this limit, L{QueueOverflow} is raised synchronously. L{None} for no limit. @ivar backlog: The maximum number of L{Deferred} gets to allow at one time. When an attempt is made to get an object which would exceed this limit, L{QueueUnderflow} is raised synchronously. L{None} for no limit. NcCsg|_g|_||_||_dSr7)rpendingsizebacklog)rFr r rrrrH%szDeferredQueue.__init__cCs|j|dS)a Remove a deferred d from our waiting list, as the deferred has been canceled. Note: We do not need to wrap this in a try/except to catch d not being in self.waiting because this canceller will not be called if d has fired. put() pops a deferred out of self.waiting and calls it, so the canceller will no longer be called. @param d: The deferred that has been canceled. Nrrerrr _cancelGet,s zDeferredQueue._cancelGetcCsL|jr|jd|n.|jdks4t|j|jkrB|j|ntdS)zq Add an object to this queue. @raise QueueOverflow: Too many objects are in this queue. rN)rrtr!r rr rJr)rFobjrrrput;s zDeferredQueue.putcCsV|jrt|jdS|jdks0t|j|jkrLt|jd}|j||St dS)a3 Attempt to retrieve and remove an object from the queue. @return: a L{Deferred} which fires with the next object available in the queue. @raise QueueUnderflow: Too many (more than C{backlog}) L{Deferred}s are already waiting for an object from this queue. rNr) r r%rtr rrr r rJrrerrrgetIs   zDeferredQueue.get)NN)rrrrrHr rrrrrrrs  rc@seZdZdZdS)AlreadyTryingToLockErrorz{ Raised when L{DeferredFilesystemLock.deferUntilLocked} is called twice on a single L{DeferredFilesystemLock}. Nrrrrrr^src@s0eZdZdZdZdZdZdddZd ddZdS) DeferredFilesystemLockai A L{FilesystemLock} that allows for a L{Deferred} to be fired when the lock is acquired. @ivar _scheduler: The object in charge of scheduling retries. In this implementation this is parameterized for testing. @ivar _interval: The retry interval for an L{IReactorTime} based scheduler. @ivar _tryLockCall: A L{DelayedCall} based on C{_interval} that will manage the next retry for acquiring the lock. @ivar _timeoutCall: A L{DelayedCall} based on C{deferUntilLocked}'s timeout argument. This is in charge of timing out our attempt to acquire the lock. r3NcCs0tj|||dkr&ddlm}|}||_dS)z @param name: The name of the lock to acquire @param scheduler: An object which provides L{IReactorTime} Nr)reactor)r FilesystemLockrHZtwisted.internetr _scheduler)rFnameZ schedulerrrrrrH|s  zDeferredFilesystemLock.__init__csRjdk rttdSfddtfddfddS) a( Wait until we acquire this lock. This method is not safe for concurrent use. @type timeout: L{float} or L{int} @param timeout: the number of seconds after which to time out if the lock has not been acquired. @return: a L{Deferred} which will callback when the lock is acquired, or errback with a L{TimeoutError} after timing out or an L{AlreadyTryingToLockError} if the L{deferUntilLocked} has already been called and not successfully locked the file. Nz/deferUntilLocked isn't safe for concurrent use.csVjd_jdk r4jr4jd_rHdn |dS)z Cancel a L{DeferredFilesystemLock.deferUntilLocked} call. @type reason: L{failure.Failure} @param reason: The reason why the call is cancelled. N) _tryLockCallrW _timeoutCallr^lockr!r&reason)r$rFrr _cancelLocks   z._cancelLockcs tSr7)rr4)rrrz9DeferredFilesystemLock.deferUntilLocked..csr4jdk r"jd_d_dnNdk rpjdkrpttdjf}j |_j j _dS)Nz&Timed out acquiring lock: %s after %fs) rrrWrr!r r-rrrrb _intervalrr_tryLockr$rFr6rrr s,   z9DeferredFilesystemLock.deferUntilLocked.._tryLock)rr'rr )rFr6rrrdeferUntilLockeds z'DeferredFilesystemLock.deferUntilLocked)N)N) rrrrrrrrHr!rrrrrfs  rrr)N)F)F)OrZ __future__rrrrrBrrsysrr functoolsrZ incrementalr Ztwisted.python.compatr r Ztwisted.pythonr r Ztwisted.loggerrZtwisted.python.deprecaterrZtwisted.python._oldstylerrrrrrrr%r'r+r1r6r9r;r<rrwrqr rZrr@rrrrrrrrr BaseExceptionrrsrrrrrrrrrrrrrrr__all__rrrrs     "   6-3  "8 B p&M+?LKf