Can't block for a Deferred in Twisted

Date: 2008-10-20 | Modified: 2008-10-27 | Tags: concurrency, python, twisted | 1 Comment

Despite the existence of the promising waitForDeferred/deferredGenerator and the newer inlineCallbacks, it appears there is no way to block while waiting for a Deferred. Brian Granger described the problem on the Twisted mailing list:

I have a function that returns a Deferred. I need to have the result of this Deferred returned in a (apparently) blocking/synchronous manner:
def myfuncBlocking():
  d = myfuncReturnsDeferred()
  ...
  result =

  return result
I need to be able to call this function like:
result = myfuncBlocking()
The question is how to get the result out of the Deferred() and make it *look* like myfuncBlocking() has blocked.

glyph provided the succinct answer (as well as an interesting commentary on using Twisted the wrong way).

This issue has been discussed repeatedly - long story short, it's just a bad idea.

Hmmm, maybe learning Twisted will be harder than I thought.

Update 2008-10-20: Marcin Kasperski wrote a good example comparing raw deferreds, deferred generators, and inline callbacks.

Twisted web POST example w/ JSON — posted 2010-08-25
Quick notes on trying the Twisted websocket branch example — posted 2010-05-24
Running a Twisted Perspective Broker example with twistd — posted 2008-10-27
Twisted links — posted 2008-10-21
Running functions periodically using Twisted's LoopingCall — posted 2008-10-14

Comments

#1 Erik commented on 2012-08-20:

You can easily use Queue.Queue to turn a Defrred call into a blocking call:

def block_on(d, timeout=None):
    q = Queue()
    d.addBoth(q.put)
    try:
        ret = q.get(timeout is not None, timeout)
    except Empty:
        raise Timeout
    if isinstance(ret, Failure):
        ret.raiseException()
    else:
        return ret

make sure you never make blocking calls from the reactor thread though.