5 ms·
I made lots of micro-edits while thinking about this, let me try and combine to something more cohesive... Also of note is that I use Twisted in my day to day.
by YZF 13y ago
I made lots of micro-edits while thinking about this, let me try and combine to something more cohesive... Also of note is that I use Twisted in my day to day.
In common Twisted programming a deferred can not fire before you add the callback since the only way a deferred can fire (for typical async work!) is through the proactor (which Twisted calls the reactor) and you haven't yet returned control to the proactor. The bug comment seems to imply a more fundamental issue but if Twisted had such a fundamental bug it just wouldn't work.
I think Guido's point is that you can still potentially add callbacks to a Deferred that has fired which opens an opportunity for bugs. He has a point there but you don't usually use Deferred that way. Generally when you add callbacks you know the Deferred hasn't fired yet.
I think Twisted isn't that bad but there are a few things I dislike about it: It's harder than I'm used to to combine synchronous and asynchronous. Twisted is "contagious".
Because the Twisted application "wants" to be single threaded, everything goes through the reactor and therefore everything has to be pure async. You can deviate from that but it has some amount of built in inflexibility. If I compare with C++ Boost::Asio there's much better multi-threading support in Asio with its worker threads (which is important for scaling across cores) and you can do more interesting combinations of async/sync IMO.
Twisted's @inlinecallbacks decorator allows you to write linear looking code you it also encourages a too-serialized way of doing things. The power of asynchronous is to do things in parallel but in Twisted you often either end up with very hard to follow "pure" asynch code or less efficient "chained" asynch code. At least that's my experience.
I personally prefer having callbacks to the Deferred returns because very often the only thing you do with a Deferred is add callbacks and it makes your code messier, more clutter/boilerplate. Returning Deferred allows for the "generator" style @inlinecallbacks which actually can be an anti-pattern from a performance perspective and also many people find it confusing to read... Explicitly chaining callbacks is what I prefer. To elaborate on callback vs. Deferred you can relatively easily build more complex structures of any kind you wish over the callback when you need it. In a sense it's the most basic way of doing things. I like basic :-)
Related to combining async and sync styles you end up seeing things like isinstance(ret, Deferred). There's also maybedeferred for dealing with async vs. sync. Still not great.
- rdtsc 13y ago> it's like Twisted is contagious. I can't quite put my finger on it. I can. Twisted is viral (not in a good way always). Once you start using Twisted you are doomed and stuck in a parallel reality of Twisted-only libraries forever. I have used Twisted professionally for 4+ years, this is not something I just read on a blog. You first say "Oh, I know look at the cute example of parsing a line protocol with it, let my try that. That was easy, I should adopt it". And then it got you. You want to fetch data from a particular database. That's easy, github has a project in Python that does it. Oh but you need it to return a deferred now. You can't simply make a connection and block because it blocks the reactor. This library is no good, you either have to pray someone who knows Twisted wrote a Twisted version of it, or you have to write your own. As you pointed out, you can use code to combine threaded and deferred code but it is not fun or easy. Make sure you think twice before adopting it. BTW the best way to learn it is to use it. Don't read about it. Reading about it won't help much. It is one of those subjects, like riding a bike if you don't know how. Once you get it you get it and it becomes easy. What is not easy is library fragmentation. You have been down-voted and perhaps I'll be too for my negativity. But I am worried about infatuation with Twisted and Promises in recent (last year and on) Python design emails and moving them more to be part of the standard library. I think this is a not healthy move for Python and a step forward. I'll speak my mind and if you feel this is too negative, just downvote, I'll understand, but I think it has to be said. They are copying what they are seeing in other language hint: node.js (Javascript). I feel there is a bit of jealousy, Python is maturing, its last exciting thing was 3.0. PyPy has been relegated to the magic academic research corner (unfairly I think). But, node.js and Javascript on the server is not successful because it has a callback style of concurrency. There are other reasons. So copying that aspect won't make Python more appealing. Heck, Python had an awesome callback library for ages -- Twisted, when Javascript was still use to make buttons flash and Java applets were the hot thing. Did it take off? Somewhat, but not really. It was famously hard to grok. The freshest breath of air so far in practical Python use on the server is using greenlet. But that also has been pushed to sort of "ah it is dirty, we don't like, how will you ever know if your code switches co-routines". Well it is not perfect but it is most practical and performant to date. I wish that became part of core Python. I wish they copied more from Go, Rust or Erlang rather than a callback style concurrency with some extra sauce on top. A Anyway, ending my rant here, and hoping I didn't upset too many people already.