3 ms·

I first came across Twisted Python and Deferreds years ago, and it was my first introduction to the world of asynchronous programming. It was a little bit diffi

I first came across Twisted Python and Deferreds years ago, and it was my first introduction to the world of asynchronous programming. It was a little bit difficult to wrap my head around the intricate braid of callbacks and errbacks, but once I realised they mapped directly onto sequential statements and try/except in synchronous code, I got a lot done. In the intervening years, I've seen a lot of projects re-implementing asynchronous programming (A perfectly valid idea, Twisted certainly has enough backwards-compatibility warts to warrant an occasional rebuilding), but the early attempts used bare callbacks for everything, which I found frustrating because I felt Deferreds were clearly a better system. Later attempts introduced "Futures" or "Promises" which were slightly better than bare callbacks but did silly things like pass the same initial result to every registered callback instead of being composable. These almost-but-not-quite-entirely-unlike-Deferreds were even more infuriating, in a "somebody is wrong on the Internet" kind of way. I'm really glad to see Guido picking the best ideas out of Twisted's Deferreds and trying to make them accessible to a professional-developer audience. If your language/runtime doesn't support some kind of co-routines, then your only hope for dealing with asynchronous code to tame callbacks, and Deferreds are the best model I've come across for doing that. (Guido mentions he won't be using Deferreds or Deferred-analogues in his Tulip async framework, for undisclosed reasons. I'm betting those reasons are that modern Python does have workable co-routines, so callbacks and callback-taming aren't necessary.)

lazyfunctor 13y ago
I am new to this. Can you give me any pointers to differences between promises and deferreds. I have some idea about promises in JS. http://promises-aplus.github.io/promises-spec/ http://promises-aplus.github.io/promises-spec/ How is twisted's deferred different?
- thristian 13y ago
  Just the first difference that caught my eye: section 3.1 of the document you linked to says that once fulfilled or rejected, a Promise must not change to any other state, and its value may not change. In Guido's exposition of Deferreds, under the heading "Idea 2: Pass results from callback to callback", he says: The most interesting bits are the last two lines: the result of each callback is passed to the next. This is different from how things work in concurrent.futures and Tulip, where the result (once set) is fixed as an attribute of the Future. Here the result can be modified by each callback. This enables a new pattern when one function returning a Deferred calls another one and transforms its result, and this is what earns this idea three stars.
- noelwelsh 13y ago
  I think this distinction should be framed in terms of Python's futures, not futures per se. Python's futures (http://docs.python.org/dev/library/concurrent.futures.html#concurrent.futures.Future http://docs.python.org/dev/library/concurrent.futures.html#c...) are very limited. They lack the equivalent of the then method in Javascript promises. This means there is very limited ability to compose futures in Python. Twisted's Deferred allow some kind of composition, though IMHO it is rather messy. You'll note in Guido's emails he says Python's futures are based on Java's. Java's futures (I can say from experience) are a completely useless crock. All you can do is poll them, which makes the whole point of asynchronous operation moot (http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/Future.html http://docs.oracle.com/javase/6/docs/api/java/util/concurren...) Javascript's promises are a better implementation, as are those in Scala and Haskell.
penguindev 13y ago
> that modern Python does have workable co-routines I think its called 'yield from'. Not sure if I'm sold on the name, but surprised nobody has mentioned it yet. From what I've seen, it looks really nice. A comment on the post has an example of it: def sync_and_read_bookmarks(): if (yield from sync_bookmarks()) > 0: return (yield from read_bookmarks())

lazyfunctor 13y ago

I am new to this. Can you give me any pointers to differences between promises and deferreds. I have some idea about promises in JS. http://promises-aplus.github.io/promises-spec/ http://promises-aplus.github.io/promises-spec/ How is twisted's deferred different?

thristian 13y ago
Just the first difference that caught my eye: section 3.1 of the document you linked to says that once fulfilled or rejected, a Promise must not change to any other state, and its value may not change. In Guido's exposition of Deferreds, under the heading "Idea 2: Pass results from callback to callback", he says: The most interesting bits are the last two lines: the result of each callback is passed to the next. This is different from how things work in concurrent.futures and Tulip, where the result (once set) is fixed as an attribute of the Future. Here the result can be modified by each callback. This enables a new pattern when one function returning a Deferred calls another one and transforms its result, and this is what earns this idea three stars.
noelwelsh 13y ago
I think this distinction should be framed in terms of Python's futures, not futures per se. Python's futures (http://docs.python.org/dev/library/concurrent.futures.html#concurrent.futures.Future http://docs.python.org/dev/library/concurrent.futures.html#c...) are very limited. They lack the equivalent of the then method in Javascript promises. This means there is very limited ability to compose futures in Python. Twisted's Deferred allow some kind of composition, though IMHO it is rather messy. You'll note in Guido's emails he says Python's futures are based on Java's. Java's futures (I can say from experience) are a completely useless crock. All you can do is poll them, which makes the whole point of asynchronous operation moot (http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/Future.html http://docs.oracle.com/javase/6/docs/api/java/util/concurren...) Javascript's promises are a better implementation, as are those in Scala and Haskell.

penguindev 13y ago

> that modern Python does have workable co-routines I think its called 'yield from'. Not sure if I'm sold on the name, but surprised nobody has mentioned it yet. From what I've seen, it looks really nice. A comment on the post has an example of it: def sync_and_read_bookmarks(): if (yield from sync_bookmarks()) > 0: return (yield from read_bookmarks())