10 ms·

Twisted Python's lead hacker discusses Tornado

101 points by thristian 17y ago

davidw 17y ago
Nice article. As someone who has been involved with open source for a while, I think one of the most annoying things is indeed when people don't even tell you about their problems. I'm already doing work for you for free; I don't have money to go do market research, so I'm counting on you to step up and say something if there's a problem.
acangiano 17y ago
He makes excellent points, but let's compare the two scenarios for a moment. A) Helping out Twisted, they'd have gotten: * Kudos from the Twisted community for their contributions to the project; * Less code required. B) Creating Tornado, they got: * To avoid other people's code they didn't like in the first place. * All the credit. They are the authors of the framework, not just contributors; * A fresh start. A project designed to their needs; * Ability to lead the project in the direction they like; * No stigmas attached (e.g., "Twisted is too complex"); * Major kudos from a large community of developers interested in "real-time" web services and applications; * Tons of free publicity to the project and FriendFeed/Facebook's open source initiatives.
- pvg 17y ago
  This is deeply silly. More fame and glory? So one day, Bret Taylor leaned back, rubbed his hands and whispered to himself 'Yesss, I shall write Tornado for fame and glory. Before I start I must briefly consider how I'll keep all the screaming teenage fans away and what to wear at the ticker-tape parade'. Or perhaps he just wanted to solve a problem he and his team had. And having done that, was nice enough to open source the solution.
  - acangiano 17y ago
    I removed my "fame and glory" comment as it was misleading and didn't truly represent what I think. I'm not arguing that he/they premeditated it and did it for the glory. Their priority was getting FF done, undoubtedly. However, there are concrete advantages (and visibility) deriving from their choice.
- Confusion 17y ago
  To avoid other people's code they didn't like in the first place. 'Not Invented Here' syndrome. A bad reason. A fresh start. A project designed to their needs That's shortsighted. Maybe the short term goals had different accents, but in the long run, Twisted and Tornado have the exact same goals. Major kudos from a large community of developers interested in "real-time" web services and applications For rewriting something that already exists and not doing an significantly better job and arguably a worse job? Applying those resources to Twisted would also have yielded this result. Tons of free publicity to the project and FriendFeed/Facebook's open source initiatives. Also belongs under A. Large contributions to OSS would yield that same effect. A fresh start. A project designed to their needs Ability to lead the project in the direction they like You are assuming, without any reason, that they wouldn't have had that chance with Twisted. If that wouldn't work out, they could easily have forked Twisted, effectively taking over the project.
  - encoderer 17y ago
    "'Not Invented Here' syndrome. A bad reason." That seems a little strawman-ish, honestly. On the contrary, they embraced the technology and architecture of the app. What's wrong with looking at something like this.. an important infrastructure piece... liking the idioms and architecture but having issues and thinking .... We have a benefit the Twisted guys didn't have... we have prior art... a map of this territory ? I don't think anybody here has a strong conviction that FriendFeed was _right_. But you and others seem to have a strong convication that they were wrong, and all I think....and I think others here too.... all I think is that it is silly to attack them and be so certain they were wrong. These are talented developers. Smart guys. With a little bit of business talent, too.. All of us have been in complex project cycles. Who are you or anybody to pick them off like they somehow caused somebody harm? Why the intensity? What gives?
  - scott_s 17y ago
    When the invention in question is core to your product, sometimes it does make sense to do it yourself. See Joel Spolsky's essay "In Defense of Not-Invented-Here Syndrome": http://www.joelonsoftware.com/articles/fog0000000007.html http://www.joelonsoftware.com/articles/fog0000000007.html
pvg 17y ago
There seems to be a rather unrealistic assumption that people who don't like your product (be it website, framework, sandwich) owe you well-reasoned feedback. It's a great and useful thing to get but the fact is, most people are interested in solving their problem and unless they are quite sure your product is the best way to solve their problem, they'll simply move on. Perhaps this is an artifact of how seriously the Twisted people and their 'lead hacker' take their project. But the twisted-centric view of the world leads to petulant posts full of odd statements: My main point here is that if you're about to undergo a re-write of a major project They wrote Tornado, they didn't 'undergo a rewrite of Twisted'. For a tiny fraction of the effort invested in Tornado, FriendFeed could have worked with us to resolve many of the issues creating that chaos. Interfacing with an another team to solve problems in a framework they didn't like (even if just aesthetically) would have probably taking significant resources for close to zero return. There's quite a bit more in the same vein. It's wonderful that the Twisted people care. But perhaps a little too much.
- Confusion 17y ago
  The assumption isn't that they owe you feedback: the assumption is that it was not in their best interest to create another instance of something that already exists. That assumption is supported by the fact that Tornado has the exact same goal as Twisted. Effectively, it is a rewrite, especially if they have obviously looked closely at Twisted's code for inspiration. A CTO wasted money here.
  - jnoller 17y ago
    No, a CTO didn't waste money; he got exactly what he needed, cleaner, simpler. No - it doesn't handle every single edge case, nuance, platform - but it does exactly what they needed. They didn't need to spend time patching and grokking twisted, or needing to deal with any rough edges to ramping personnel up on it. They had a job to do, and they did it. Now that it's out in the wild, maybe someone will cross-pollinate, maybe changes will go in both projects. But the good thing is that they got their job done.
    - 17y ago
tdavis 17y ago
One thing to remember about Twisted is that it is basically worked on by two people. It is an enormous code base for those two to maintain, develop on, and document at the same time, even if they are ridiculously smart (which they are). I gave kudos to Tornado, but Twisted deserves some appreciation too -- and contributors! I tried to take twisted.web into the present (the client anyway), but I unfortunately wasn't sufficiently TDD to get it contributed. Perhaps next time!
- codexon 17y ago
  I submitted a patch to Twisted to fix their SOCKS implementation. It was "accepted" after a month, and by accepted it means put into a branch where it will take up to a year to be incorporated into the mainline depending on whether the few maintainers feel like taking a look at it.
- andrewtj 17y ago
  Two people is news to me — their website lead me to believe otherwise: http://twistedmatrix.com/trac/wiki/TwistedMatrixLaboratories http://twistedmatrix.com/trac/wiki/TwistedMatrixLaboratories
moe 17y ago
I agree that FF should have asked and should have used twisted under the hood. I hope the porting efforts will be successful and eventually replace the original Tornado impl. I cannot help, though, but yet again in this blog post notice the unfortunate, defensive tone that is so omnipresent in the twisted community. The post starts out by (politely and rightfully) pointing out FriendFeed's communication mistake and then goes on to describe how green the grass would be in twisted land. What I miss is a bit more critical self-reflection about the reasons for FF skipping twisted, even without FF spelling them out in detail. It's not likely that FF found completely new and unseen problems with twisted. I'm more inclined to believe that they simply stumbled over the standard, long-known twisted problems; first and foremost the lack of documentation and (probably) the verbose coding-style that many parts of twisted demand. I have the highest respect for the effort that went into twisted and for their often cited code quality. But they need to realize that they can't display a "your own fault if you're not smart enough for this"-attitude towards documentation and at the same time mock anyone who doesn't want to put up with that. Like any OSS project twisted is ofcourse not obliged to anything. But neither are people like the FF guys obliged to choose twisted.
- rg123 17y ago
  I 2nd this completely re: Twisted documentation. When Twisted's communication via their documentation has been so lacking/off-putting/etc., it is hard to feel sympathy for a Twisted guy complaining about the lack of communication from people who chose a different route.
andrewvc 17y ago
Well, if this benchmark from yesterday is accurate, then Twisted can't handle as many concurrent connections as Tornado. I'd say that's a good reason not to use Twisted. http://www.apparatusproject.org/blog/2009/09/twisted-web-vs-tornado-performance-test/ http://www.apparatusproject.org/blog/2009/09/twisted-web-vs-...
- roder 17y ago
  If you read the comments in the blog, it's likely just related to the platform. I did the test on my mac, which certainly isn't the final say... I'll redo the test today different hardware today.
roder 17y ago
I thought that glyph's response was appropriate and on the mark. It seemed to me that Bret had taken a few jabs at Twisted without really providing quantitate evidence. Particularly the comments about being "demo-quality" and "tons of bugs". It would have been nice to see innovation happen collaboratively between FriendFeed and the Twisted team. Everyone would have benefited more IMHO. Twisted has certainly done a lot for event-based architecture. As Glyph points out, the use of Deferreds have made their way into javascript frameworks like dojo. To me, that's credible and proof of twisted's impact in the development community.
- boucher 17y ago
  I really think that calling Deferreds a "de-facto standard" in JavaScript is a huge stretch. I'm having trouble pinning down exactly what makes Deferreds different from any asynchronous callback system, but it seems to be the chain of callbacks?
  - dlsspy 17y ago
    One of the frustrating differences I ran into is the callback mechanism in tornado. You can't attach more than one callback, and the one kind of callback does two different things (meaning the callback has to determine whether or not the call was successful before it does its work). To me, this makes things more complicated. What if I want to use the same error handling code for all bits of code (really common for me is `d.addErrback(log.err)` to just log it), but each successful call has to do something different? Twisted deferreds also allow you to compose chains. This is a somewhat trivial example, but by having the callback do both success and error cases, I had to build an adaptor to pass to callbacks to include the "is this a failure" information. The easy way to do that is to create a wrapper class and construct it one way or another based on whether a success or error occurred. I ended up just composing it like this: deferred.addCallback(lambda x: Response(body=x)) deferred.addErrback(lambda e: Response(error=e)) deferred.addCallback(lambda r: callback(r, *args, **kwargs)) The first callback and errback create a Response (wrapper) object that contains either a body or an error. The second callback receives the response that was made by one of the above and calls the initial callback. This is sort of unrolling complexity and sweeping it away. The real problem is that you can only have one callback in tornado (and you can't easily chain them). That means that what actually happens when this deferred's callback fires is that it ends up calling the original callback, which is an inner wrapper function inside of another function that calls a curried function that calls your original callback -- and you have to create these things by name when you're building out your auth class. To me, that's complicated. deferreds seem like there's more involved than you get with a simple callback until you actually try to use the simple callback. The interesting properties of deferreds are 1) You can add multiple callbacks and errbacks and build chains as complicated as your system needs including composing common cases (such as success and error counters). 2) If a callback (or errback) returns a deferred, you spin off another dimension of the chain. #2 is why the above code and about twenty lines of web.py exist for a single use case. Here's a concrete example (close to something I do in the real world™): Imagine a simple twitter API. You go grab a tweet and then give it to something else: `t.grabTweet(something).addCallback(processTweet)` Now someone adds a requirement that between `grabTweet` and `processTweet` you want to perform short URL expansion. In twisted, I'd just toss an `addCallback(expandURL)` in the middle and call my awesomely well-tested URL expander -- which makes an async http call to another service and doesn't call the next callback in the chain until that service returns the data I need to fill satisfy the request. And then after I did that, I found that the URL expansion stuff I was using was blowing up the upstream and causing me to fail to expand some, so I added async cache support to those inside the API (not as an additional toplevel callback). For this new step, the toplevel API doesn't change, but now I can do more async stuff that may lead to further async stuff without having to write additional glue code.
smoody 17y ago
I understand the response at some level, but I also totally get the idea of making things work the way one's mind works. It makes coding much more enjoyable and speeds the process of app development once the core work is done. And it was nice of the author to do some QA work for Tornado, too. Hopefully he'll go the extra step and submit those bugs he found as tickets so that he's walking the walk and not just talking the talk. :-)
rams 17y ago
At one time the Twisted guys were accused of walking away instead of trying to improve Python's async module. History repeats.
- nostrademons 17y ago
  I find it interesting that nearly all the comments here are on Tornado vs. Twisted rather than the larger issue of communicating vs. coding that the author brings up. There've been a lot of times - most recently yesterday - when I've hacked up a solution, mailed it off, and got back a response of "I just did this a couple days ago. With a one-line change, my code could've covered your needs too." And then of course I'm like, "Oh. Shit. Let's modify your code instead and I'll throw away my changelist." I get the feeling this happens a lot outside of my own personal/professional sphere as well. Google's famous for its NIH syndrome. Python's got about 5 different competing web frameworks. Lots of people still write their own machine code generators instead of using LLVM. I'm kinda curious how much coding people are willing to do to save one e-mail. Ironically, the folks who insist on building their own stuff are often right. Google's certainly done well for itself, and it's great that we can use Django instead of Zope now. So maybe the problem is that communication really is so expensive that people are better off rewriting large chunks of code to avoid it. But if that's true, we should be looking for ways to reduce the cost of communication, rather than glorifying the folks who go to heroic measures to avoid it. It's curious - yesterday, Coders at Work was at the top of the hotlist, and most of the comments said, "I'd rather learn about the business & management aspects than the technical aspects." Yet here's an interesting business & management problem, but everyone's commenting on the technical aspects.
  - deleted 17y ago
    [deleted]
  - lacker 17y ago
    I think the author is misinterpreting what's going on. It's not that it's too hard to communicate with the Twisted folks, it's that the FriendFeed folks didn't think Twisted was good enough. Twisted is too buggy and too complicated.
  - 17y ago
johnbender 17y ago
One reason I wouldn't have reached out, as a private company, is tipping my hand to competitors. Maybe an NDA would solve that?
n8agrin 17y ago
An observation: Python coders seem to believe that when someone has solved a problem in Python and open sourced that problem every Python coder must use their solution or else they are somehow doing an injustice, perhaps even undermining the community! On the contrary, 1) Friendfeed can do whatever they want. There is no part of the Python license that says you must use Twisted or else. 2) I doubt FF's statements really caused damage to Twisted's user base, just as we use Cherrypy at work and didn't drop it on Thursday because of Tornado. 3) All open source contributions are a Good Thing. If Twisted devs are really feeling screwed FF is basically saying "here's how we did it, use it if you want". Sounds pretty win win to me. Instead of complaining and accusing I would suggest Twisted devs take the publicity opportunity to show why Twisted is still great, benchmarks, what the do better than Tornado and what they can learn from it.
- cturner 17y ago
  An observation: Python coders seem to believe That's a poor generalisation, and I assume you've built it on the back of a single observation. There are dozens of python template libraries that solve problems with different levels of complexity, and there's no animosity over it. Indeed, one of the reasons I like python is that there's a strong standard library, and then a healthy diversity of tools so you can mix and match to get what you want. I doubt FF's statements really caused damage to Twisted's user base They implied that it had bottlenecks. Your suggestion in the last paragraph might cause benefit. General comment on difficulties learning twisted - I found it difficult to grasp until I'd practically reimplemented the patterns by accident in the course of writing my own client/server application. I got stuck on a concept at one point in my non-twisted socket application, and the guys on #twisted helped me past it. For those who are interested, the O'Reilly book is obsolete but a better starting point than the online docs.
  - n8agrin 17y ago
    That's a poor generalisation, and I assume you've built it on the back of a single observation. On the contrary, my observation is based on comments I've read in many Python community forums, especially when applied to Python vs (your other high-level language of choice). For example, I've read many comments to the effect of "I like Python over (Ruby, Perl, etc) because it promotes one right way of doing things and having one package for handling problem X." I am not stating that this belief is a Python language requirement, nor do I believe that this observation adheres to all Python coders so yes, it is a generalization. But my generalization is certainly not based on a single observation. Finally, I will gladly play the devil's advocate and agree that there is little way to prove this generalization, short of spending several days collecting emails on Python mailing lists, which is why it is offered as an observation only. It is an observation of the practices of a community not something codified into a PEP, at least as far as I know based on my limited research. I completely agree with the rest of your comment and value your critique. Python is a great language that provides a diverse set of tools, which is why I also choose it to handle many tasks.
lacker 17y ago
The Twisted people don't seem to get that their framework is too complicated for many uses. You can't just work on someone else's framework and make it simpler, if they don't agree that it should be simple in the first place.
- dlsspy 17y ago
  "You can't just work on someone else's framework and make it simpler" Are you suggesting that python is more complex to use than x86_64 assembler? Similarly, when I removed code from tornado to make it work with twisted, I neither tornado more complicated to use, nor did I introduce any limitations. The "too complicated for many uses" thing kind of bugs me, though. It's the same argument people use to use for postgres vs. mysql. Something along the lines of "it has features I don't need" (where need probably means "understand" or "need today"). That's where tornado left me. What you call complication, I call features. I use some of these features. For example, I'd be interested in knowing how something like my tdf web server could be written in tornado. It attaches processes to resource paths, captures and transforms the output from these processes (in a stream) and streams replicates of that data to whatever connections come in. The answer, of course, involves reimplementing reactor.spawnProcess on top of ioloop. Then you find some of the subtleties contained therein. Then you begin to understand why it seemed complicated to begin with.
  - evgen 17y ago
    > Are you suggesting that python is more complex to use than x86_64 assembler? No, but twisted is much more complicated than regular python. The inversion of control and the fact that until quite recently it was almost impossible to write code for using the framework that was easy to read/understand by experienced python ciders who were not familiar with the idims and conventions used by the project. Twisted generates more WTFs per minute than any other bit of python kit. Some of this is due to a tradition of poor documentation within the project and some due to the nature of the code. Twisted is a complicated system, and sometimes this complexity is not worth embracing.
crad 17y ago
I like the idea of the performance and simplicity of Tornado, as I primarily use web.py. After reading this blog entry I decided to give Twisted another look. It took me about 2 minutes to get to the info about the web module and the examples were not very clear. I've walked away wondering a few things: If I use twisted, should I use web or web2? How do I map URI's to functions? When I go to the tornado homepage, I scroll and in < 1 minute I see how I would code for it and can be off and running. To develop web apps, I shouldn't have to spend time figuring out an async network communications framework, I should be worried about programming my app. My perception from the Twisted site is that I would need to spend time figuring out the twisted way of doing things, then figure out the web add-on way of doing things, then worry about my app. With Tornado, I'm very close to a drop in replacement for web.py and I can be off to the races.
- dlsspy 17y ago
  Your perception of twisted may be as it is because you can do SO MUCH with twisted. If it's a web.py replacement you want, it doesn't add complexity. Take a look at the diff to go from a friendfeed ioloop to a twisted reactor loop (with my adaptor) in tornado: http://pastebin.com/f707e5ae9 http://pastebin.com/f707e5ae9 You replace two imports with two other imports, and three lines of startup code with three other lines of startup code. Your app doesn't change.
  - scott_s 17y ago
    Which is the same as saying it's a failure of documentation and explanation.
  - crad 17y ago
    I was eyeing up the code earlier for this. This is great and is a good counter for my argument about Tornado's near drop in replacement. However, that's not my main point. Ultimately, I think Twisted, due to the fact it does so much, is intimidating. The documentation lacks succinct examples of how to implement the very basic things people want to implement. The documentation to figure out how to make an app is buried under at least 3 clicks and even then isn't clear. The comments, when there are comments, in the examples don't truly explain the why of the actions just that they are there. If twisted wants to be considered as a serious contender as foundation to develop web apps on, they need to take a look at how important very good documentation is to Django's success or how the simple, uncommented example at http://www.tornadoweb.org/ http://www.tornadoweb.org/ is more clear than the first bit of web example code at twisted: http://twistedmatrix.com/projects/web/documentation/examples/silly-web.py http://twistedmatrix.com/projects/web/documentation/examples.... I'm not knocking twisted, I'm trying to explain why when I went around evaluating what I would use in my projects, I did not choose Twisted. If you find my decisions and observations valid or not is up to you.
calambrac 17y ago
Twisted has always been an "emperor has no clothes" project to me, a project that's no doubt very useful but at the same time seems to revel in its unnecessary complexity (just read its documentation - how many times do you really have to tell us that something might seem overly complicated but is worth it for its flexibility?) I'm honestly glad to see a no-bullshit alternative that doesn't have that same sense of worshipping complexity as a sign of quality.
- enneff 17y ago
  That viewpoint seems quite narrow to me. The documentation of Twisted attempts to introduce several event-driven programming patterns. Most people have a 'wtf' moment or two when they come across design patterns that approach things seemingly backwards. The further I got into my Twisted app, the more I found myself reaching for things and finding that were already there for me. It was a great experience. You should try it!
  - calambrac 17y ago
    I've done plenty of work with Twisted. My complaint was not that it lacked functionality, my complaint was that the culture of the project favors 'flexibility' over everything else. It's like the C++ of networking libraries. I'm sure there are plenty of times and places where its approach is exactly what you need, but it's just rarely a good fit for what I need.
Tichy 17y ago
It sounds very good, but I must admit I immediately remembered lots and lots of comments from open source developers that amounted to "not happy with our project (a ka 'free toilet')? well duh, hack it yourself, we owe you s*". Obviously not the fault of the Twisted developer, but I am not surprised if people actually prefer to hack their own stuff.
paul 17y ago
The great thing about open source is that people are free to share ideas and code. If the Twisted team likes parts of Tornado, they are free to replicate those parts back into Twisted while ignoring the parts that they don't like. Meanwhile, users are free to choose whichever approach suits them better. Twisted is no doubt more complete in terms of protocols and such, but many people also value simplicity. One important point that I think has been overlooked here is that the core of Tornado is only a few hundred lines of code. For me, the fact that I can actually read _all_ of the code in just a few minutes is very valuable. I'm sure many others don't care though, which is why it's great that we live in a world where there doesn't always have to be a single "right" answer.
- dlsspy 17y ago
  "Twisted is no doubt more complete in terms of protocols and such, but many people also value simplicity. One important point that I think has been overlooked here is that the core of Tornado is only a few hundred lines of code." I think this is a bit misleading. The twisted port involved removing almost 1,400 lines of code and didn't change the tornado API. That is, tornado is not more simple without twisted. I do understand the value of being able to read all of the code quickly, but there are other turtles below these. For me, it's very valuable to know that the part upon which I'm building is a well-tested codebase whose semantics I understand, and for which I can write tests when things begin to violate my understanding. If tornado is used in more applications, it will have to grow in complexity to meet the needs of those applications. I see tornado as having two faces. The web portions can certainly continue to grow, but the network IO handing just means that people will have to port over things that already exist within twisted, and expand it in complexity (such as the kqueue work that is in progress) to handle more low-level stuff. At some point, you can't just easily sit down and read all of it (at least, not any more easily than you can with twisted).
  - paul 17y ago
    Tornado relies on a small number of well defined system calls (such as epoll). For me, the difference between reading (and debugging) a few hundred lines of code down to the system call level, and reading (and debugging) a big library like Twisted is rather significant. For you, the difference may not be important, and that's fine. I also don't think that it's necessary for Tornado to replicate all the functionality of Twisted -- if people want Twisted, they should simply use Twisted and ignore Tornado (and hopefully the Twisted team will have borrowed the best bits of Tornado anyway).
frognibble 17y ago
There are several async programming libraries for Python including Allegra ( http://laurentszyster.be/blog/allegra/doc/ http://laurentszyster.be/blog/allegra/doc/ ), libevent-python (and other libevent bindings to python), the standard asyncore module and more. Why should Facebook use Twisted instead of one of these other libraries?
- imbaczek 17y ago
  more importantly, why should it roll it's own given a plethora of more mature alternatives?
  - frognibble 17y ago
    Perhaps there are many alternatives because they all have problems.

101 points by thristian 17y ago

Nice article. As someone who has been involved with open source for a while, I think one of the most annoying things is indeed when people don't even tell you about their problems. I'm already doing work for you for free; I don't have money to go do market research, so I'm counting on you to step up and say something if there's a problem.

acangiano 17y ago

He makes excellent points, but let's compare the two scenarios for a moment. A) Helping out Twisted, they'd have gotten: * Kudos from the Twisted community for their contributions to the project; * Less code required. B) Creating Tornado, they got: * To avoid other people's code they didn't like in the first place. * All the credit. They are the authors of the framework, not just contributors; * A fresh start. A project designed to their needs; * Ability to lead the project in the direction they like; * No stigmas attached (e.g., "Twisted is too complex"); * Major kudos from a large community of developers interested in "real-time" web services and applications; * Tons of free publicity to the project and FriendFeed/Facebook's open source initiatives.

pvg 17y ago
This is deeply silly. More fame and glory? So one day, Bret Taylor leaned back, rubbed his hands and whispered to himself 'Yesss, I shall write Tornado for fame and glory. Before I start I must briefly consider how I'll keep all the screaming teenage fans away and what to wear at the ticker-tape parade'. Or perhaps he just wanted to solve a problem he and his team had. And having done that, was nice enough to open source the solution.
- acangiano 17y ago
  I removed my "fame and glory" comment as it was misleading and didn't truly represent what I think. I'm not arguing that he/they premeditated it and did it for the glory. Their priority was getting FF done, undoubtedly. However, there are concrete advantages (and visibility) deriving from their choice.
Confusion 17y ago
To avoid other people's code they didn't like in the first place. 'Not Invented Here' syndrome. A bad reason. A fresh start. A project designed to their needs That's shortsighted. Maybe the short term goals had different accents, but in the long run, Twisted and Tornado have the exact same goals. Major kudos from a large community of developers interested in "real-time" web services and applications For rewriting something that already exists and not doing an significantly better job and arguably a worse job? Applying those resources to Twisted would also have yielded this result. Tons of free publicity to the project and FriendFeed/Facebook's open source initiatives. Also belongs under A. Large contributions to OSS would yield that same effect. A fresh start. A project designed to their needs Ability to lead the project in the direction they like You are assuming, without any reason, that they wouldn't have had that chance with Twisted. If that wouldn't work out, they could easily have forked Twisted, effectively taking over the project.
- encoderer 17y ago
  "'Not Invented Here' syndrome. A bad reason." That seems a little strawman-ish, honestly. On the contrary, they embraced the technology and architecture of the app. What's wrong with looking at something like this.. an important infrastructure piece... liking the idioms and architecture but having issues and thinking .... We have a benefit the Twisted guys didn't have... we have prior art... a map of this territory ? I don't think anybody here has a strong conviction that FriendFeed was _right_. But you and others seem to have a strong convication that they were wrong, and all I think....and I think others here too.... all I think is that it is silly to attack them and be so certain they were wrong. These are talented developers. Smart guys. With a little bit of business talent, too.. All of us have been in complex project cycles. Who are you or anybody to pick them off like they somehow caused somebody harm? Why the intensity? What gives?
- scott_s 17y ago
  When the invention in question is core to your product, sometimes it does make sense to do it yourself. See Joel Spolsky's essay "In Defense of Not-Invented-Here Syndrome": http://www.joelonsoftware.com/articles/fog0000000007.html http://www.joelonsoftware.com/articles/fog0000000007.html

pvg 17y ago

There seems to be a rather unrealistic assumption that people who don't like your product (be it website, framework, sandwich) owe you well-reasoned feedback. It's a great and useful thing to get but the fact is, most people are interested in solving their problem and unless they are quite sure your product is the best way to solve their problem, they'll simply move on. Perhaps this is an artifact of how seriously the Twisted people and their 'lead hacker' take their project. But the twisted-centric view of the world leads to petulant posts full of odd statements: My main point here is that if you're about to undergo a re-write of a major project They wrote Tornado, they didn't 'undergo a rewrite of Twisted'. For a tiny fraction of the effort invested in Tornado, FriendFeed could have worked with us to resolve many of the issues creating that chaos. Interfacing with an another team to solve problems in a framework they didn't like (even if just aesthetically) would have probably taking significant resources for close to zero return. There's quite a bit more in the same vein. It's wonderful that the Twisted people care. But perhaps a little too much.

Confusion 17y ago
The assumption isn't that they owe you feedback: the assumption is that it was not in their best interest to create another instance of something that already exists. That assumption is supported by the fact that Tornado has the exact same goal as Twisted. Effectively, it is a rewrite, especially if they have obviously looked closely at Twisted's code for inspiration. A CTO wasted money here.
- jnoller 17y ago
  No, a CTO didn't waste money; he got exactly what he needed, cleaner, simpler. No - it doesn't handle every single edge case, nuance, platform - but it does exactly what they needed. They didn't need to spend time patching and grokking twisted, or needing to deal with any rough edges to ramping personnel up on it. They had a job to do, and they did it. Now that it's out in the wild, maybe someone will cross-pollinate, maybe changes will go in both projects. But the good thing is that they got their job done.
  - 17y ago

tdavis 17y ago

One thing to remember about Twisted is that it is basically worked on by two people. It is an enormous code base for those two to maintain, develop on, and document at the same time, even if they are ridiculously smart (which they are). I gave kudos to Tornado, but Twisted deserves some appreciation too -- and contributors! I tried to take twisted.web into the present (the client anyway), but I unfortunately wasn't sufficiently TDD to get it contributed. Perhaps next time!

codexon 17y ago
I submitted a patch to Twisted to fix their SOCKS implementation. It was "accepted" after a month, and by accepted it means put into a branch where it will take up to a year to be incorporated into the mainline depending on whether the few maintainers feel like taking a look at it.
andrewtj 17y ago
Two people is news to me — their website lead me to believe otherwise: http://twistedmatrix.com/trac/wiki/TwistedMatrixLaboratories http://twistedmatrix.com/trac/wiki/TwistedMatrixLaboratories

moe 17y ago

I agree that FF should have asked and should have used twisted under the hood. I hope the porting efforts will be successful and eventually replace the original Tornado impl. I cannot help, though, but yet again in this blog post notice the unfortunate, defensive tone that is so omnipresent in the twisted community. The post starts out by (politely and rightfully) pointing out FriendFeed's communication mistake and then goes on to describe how green the grass would be in twisted land. What I miss is a bit more critical self-reflection about the reasons for FF skipping twisted, even without FF spelling them out in detail. It's not likely that FF found completely new and unseen problems with twisted. I'm more inclined to believe that they simply stumbled over the standard, long-known twisted problems; first and foremost the lack of documentation and (probably) the verbose coding-style that many parts of twisted demand. I have the highest respect for the effort that went into twisted and for their often cited code quality. But they need to realize that they can't display a "your own fault if you're not smart enough for this"-attitude towards documentation and at the same time mock anyone who doesn't want to put up with that. Like any OSS project twisted is ofcourse not obliged to anything. But neither are people like the FF guys obliged to choose twisted.

rg123 17y ago
I 2nd this completely re: Twisted documentation. When Twisted's communication via their documentation has been so lacking/off-putting/etc., it is hard to feel sympathy for a Twisted guy complaining about the lack of communication from people who chose a different route.

andrewvc 17y ago

Well, if this benchmark from yesterday is accurate, then Twisted can't handle as many concurrent connections as Tornado. I'd say that's a good reason not to use Twisted. http://www.apparatusproject.org/blog/2009/09/twisted-web-vs-tornado-performance-test/ http://www.apparatusproject.org/blog/2009/09/twisted-web-vs-...

roder 17y ago
If you read the comments in the blog, it's likely just related to the platform. I did the test on my mac, which certainly isn't the final say... I'll redo the test today different hardware today.

roder 17y ago

I thought that glyph's response was appropriate and on the mark. It seemed to me that Bret had taken a few jabs at Twisted without really providing quantitate evidence. Particularly the comments about being "demo-quality" and "tons of bugs". It would have been nice to see innovation happen collaboratively between FriendFeed and the Twisted team. Everyone would have benefited more IMHO. Twisted has certainly done a lot for event-based architecture. As Glyph points out, the use of Deferreds have made their way into javascript frameworks like dojo. To me, that's credible and proof of twisted's impact in the development community.

boucher 17y ago
I really think that calling Deferreds a "de-facto standard" in JavaScript is a huge stretch. I'm having trouble pinning down exactly what makes Deferreds different from any asynchronous callback system, but it seems to be the chain of callbacks?
- dlsspy 17y ago
  One of the frustrating differences I ran into is the callback mechanism in tornado. You can't attach more than one callback, and the one kind of callback does two different things (meaning the callback has to determine whether or not the call was successful before it does its work). To me, this makes things more complicated. What if I want to use the same error handling code for all bits of code (really common for me is `d.addErrback(log.err)` to just log it), but each successful call has to do something different? Twisted deferreds also allow you to compose chains. This is a somewhat trivial example, but by having the callback do both success and error cases, I had to build an adaptor to pass to callbacks to include the "is this a failure" information. The easy way to do that is to create a wrapper class and construct it one way or another based on whether a success or error occurred. I ended up just composing it like this: deferred.addCallback(lambda x: Response(body=x)) deferred.addErrback(lambda e: Response(error=e)) deferred.addCallback(lambda r: callback(r, *args, **kwargs)) The first callback and errback create a Response (wrapper) object that contains either a body or an error. The second callback receives the response that was made by one of the above and calls the initial callback. This is sort of unrolling complexity and sweeping it away. The real problem is that you can only have one callback in tornado (and you can't easily chain them). That means that what actually happens when this deferred's callback fires is that it ends up calling the original callback, which is an inner wrapper function inside of another function that calls a curried function that calls your original callback -- and you have to create these things by name when you're building out your auth class. To me, that's complicated. deferreds seem like there's more involved than you get with a simple callback until you actually try to use the simple callback. The interesting properties of deferreds are 1) You can add multiple callbacks and errbacks and build chains as complicated as your system needs including composing common cases (such as success and error counters). 2) If a callback (or errback) returns a deferred, you spin off another dimension of the chain. #2 is why the above code and about twenty lines of web.py exist for a single use case. Here's a concrete example (close to something I do in the real world™): Imagine a simple twitter API. You go grab a tweet and then give it to something else: `t.grabTweet(something).addCallback(processTweet)` Now someone adds a requirement that between `grabTweet` and `processTweet` you want to perform short URL expansion. In twisted, I'd just toss an `addCallback(expandURL)` in the middle and call my awesomely well-tested URL expander -- which makes an async http call to another service and doesn't call the next callback in the chain until that service returns the data I need to fill satisfy the request. And then after I did that, I found that the URL expansion stuff I was using was blowing up the upstream and causing me to fail to expand some, so I added async cache support to those inside the API (not as an additional toplevel callback). For this new step, the toplevel API doesn't change, but now I can do more async stuff that may lead to further async stuff without having to write additional glue code.

smoody 17y ago

I understand the response at some level, but I also totally get the idea of making things work the way one's mind works. It makes coding much more enjoyable and speeds the process of app development once the core work is done. And it was nice of the author to do some QA work for Tornado, too. Hopefully he'll go the extra step and submit those bugs he found as tickets so that he's walking the walk and not just talking the talk. :-)

rams 17y ago

At one time the Twisted guys were accused of walking away instead of trying to improve Python's async module. History repeats.

nostrademons 17y ago
I find it interesting that nearly all the comments here are on Tornado vs. Twisted rather than the larger issue of communicating vs. coding that the author brings up. There've been a lot of times - most recently yesterday - when I've hacked up a solution, mailed it off, and got back a response of "I just did this a couple days ago. With a one-line change, my code could've covered your needs too." And then of course I'm like, "Oh. Shit. Let's modify your code instead and I'll throw away my changelist." I get the feeling this happens a lot outside of my own personal/professional sphere as well. Google's famous for its NIH syndrome. Python's got about 5 different competing web frameworks. Lots of people still write their own machine code generators instead of using LLVM. I'm kinda curious how much coding people are willing to do to save one e-mail. Ironically, the folks who insist on building their own stuff are often right. Google's certainly done well for itself, and it's great that we can use Django instead of Zope now. So maybe the problem is that communication really is so expensive that people are better off rewriting large chunks of code to avoid it. But if that's true, we should be looking for ways to reduce the cost of communication, rather than glorifying the folks who go to heroic measures to avoid it. It's curious - yesterday, Coders at Work was at the top of the hotlist, and most of the comments said, "I'd rather learn about the business & management aspects than the technical aspects." Yet here's an interesting business & management problem, but everyone's commenting on the technical aspects.
- deleted 17y ago
  [deleted]
- lacker 17y ago
  I think the author is misinterpreting what's going on. It's not that it's too hard to communicate with the Twisted folks, it's that the FriendFeed folks didn't think Twisted was good enough. Twisted is too buggy and too complicated.
- 17y ago

johnbender 17y ago

One reason I wouldn't have reached out, as a private company, is tipping my hand to competitors. Maybe an NDA would solve that?

n8agrin 17y ago

An observation: Python coders seem to believe that when someone has solved a problem in Python and open sourced that problem every Python coder must use their solution or else they are somehow doing an injustice, perhaps even undermining the community! On the contrary, 1) Friendfeed can do whatever they want. There is no part of the Python license that says you must use Twisted or else. 2) I doubt FF's statements really caused damage to Twisted's user base, just as we use Cherrypy at work and didn't drop it on Thursday because of Tornado. 3) All open source contributions are a Good Thing. If Twisted devs are really feeling screwed FF is basically saying "here's how we did it, use it if you want". Sounds pretty win win to me. Instead of complaining and accusing I would suggest Twisted devs take the publicity opportunity to show why Twisted is still great, benchmarks, what the do better than Tornado and what they can learn from it.

cturner 17y ago
An observation: Python coders seem to believe That's a poor generalisation, and I assume you've built it on the back of a single observation. There are dozens of python template libraries that solve problems with different levels of complexity, and there's no animosity over it. Indeed, one of the reasons I like python is that there's a strong standard library, and then a healthy diversity of tools so you can mix and match to get what you want. I doubt FF's statements really caused damage to Twisted's user base They implied that it had bottlenecks. Your suggestion in the last paragraph might cause benefit. General comment on difficulties learning twisted - I found it difficult to grasp until I'd practically reimplemented the patterns by accident in the course of writing my own client/server application. I got stuck on a concept at one point in my non-twisted socket application, and the guys on #twisted helped me past it. For those who are interested, the O'Reilly book is obsolete but a better starting point than the online docs.
- n8agrin 17y ago
  That's a poor generalisation, and I assume you've built it on the back of a single observation. On the contrary, my observation is based on comments I've read in many Python community forums, especially when applied to Python vs (your other high-level language of choice). For example, I've read many comments to the effect of "I like Python over (Ruby, Perl, etc) because it promotes one right way of doing things and having one package for handling problem X." I am not stating that this belief is a Python language requirement, nor do I believe that this observation adheres to all Python coders so yes, it is a generalization. But my generalization is certainly not based on a single observation. Finally, I will gladly play the devil's advocate and agree that there is little way to prove this generalization, short of spending several days collecting emails on Python mailing lists, which is why it is offered as an observation only. It is an observation of the practices of a community not something codified into a PEP, at least as far as I know based on my limited research. I completely agree with the rest of your comment and value your critique. Python is a great language that provides a diverse set of tools, which is why I also choose it to handle many tasks.

lacker 17y ago

The Twisted people don't seem to get that their framework is too complicated for many uses. You can't just work on someone else's framework and make it simpler, if they don't agree that it should be simple in the first place.

dlsspy 17y ago
"You can't just work on someone else's framework and make it simpler" Are you suggesting that python is more complex to use than x86_64 assembler? Similarly, when I removed code from tornado to make it work with twisted, I neither tornado more complicated to use, nor did I introduce any limitations. The "too complicated for many uses" thing kind of bugs me, though. It's the same argument people use to use for postgres vs. mysql. Something along the lines of "it has features I don't need" (where need probably means "understand" or "need today"). That's where tornado left me. What you call complication, I call features. I use some of these features. For example, I'd be interested in knowing how something like my tdf web server could be written in tornado. It attaches processes to resource paths, captures and transforms the output from these processes (in a stream) and streams replicates of that data to whatever connections come in. The answer, of course, involves reimplementing reactor.spawnProcess on top of ioloop. Then you find some of the subtleties contained therein. Then you begin to understand why it seemed complicated to begin with.
- evgen 17y ago
  > Are you suggesting that python is more complex to use than x86_64 assembler? No, but twisted is much more complicated than regular python. The inversion of control and the fact that until quite recently it was almost impossible to write code for using the framework that was easy to read/understand by experienced python ciders who were not familiar with the idims and conventions used by the project. Twisted generates more WTFs per minute than any other bit of python kit. Some of this is due to a tradition of poor documentation within the project and some due to the nature of the code. Twisted is a complicated system, and sometimes this complexity is not worth embracing.

crad 17y ago

I like the idea of the performance and simplicity of Tornado, as I primarily use web.py. After reading this blog entry I decided to give Twisted another look. It took me about 2 minutes to get to the info about the web module and the examples were not very clear. I've walked away wondering a few things: If I use twisted, should I use web or web2? How do I map URI's to functions? When I go to the tornado homepage, I scroll and in < 1 minute I see how I would code for it and can be off and running. To develop web apps, I shouldn't have to spend time figuring out an async network communications framework, I should be worried about programming my app. My perception from the Twisted site is that I would need to spend time figuring out the twisted way of doing things, then figure out the web add-on way of doing things, then worry about my app. With Tornado, I'm very close to a drop in replacement for web.py and I can be off to the races.

dlsspy 17y ago
Your perception of twisted may be as it is because you can do SO MUCH with twisted. If it's a web.py replacement you want, it doesn't add complexity. Take a look at the diff to go from a friendfeed ioloop to a twisted reactor loop (with my adaptor) in tornado: http://pastebin.com/f707e5ae9 http://pastebin.com/f707e5ae9 You replace two imports with two other imports, and three lines of startup code with three other lines of startup code. Your app doesn't change.
- scott_s 17y ago
  Which is the same as saying it's a failure of documentation and explanation.
- crad 17y ago
  I was eyeing up the code earlier for this. This is great and is a good counter for my argument about Tornado's near drop in replacement. However, that's not my main point. Ultimately, I think Twisted, due to the fact it does so much, is intimidating. The documentation lacks succinct examples of how to implement the very basic things people want to implement. The documentation to figure out how to make an app is buried under at least 3 clicks and even then isn't clear. The comments, when there are comments, in the examples don't truly explain the why of the actions just that they are there. If twisted wants to be considered as a serious contender as foundation to develop web apps on, they need to take a look at how important very good documentation is to Django's success or how the simple, uncommented example at http://www.tornadoweb.org/ http://www.tornadoweb.org/ is more clear than the first bit of web example code at twisted: http://twistedmatrix.com/projects/web/documentation/examples/silly-web.py http://twistedmatrix.com/projects/web/documentation/examples.... I'm not knocking twisted, I'm trying to explain why when I went around evaluating what I would use in my projects, I did not choose Twisted. If you find my decisions and observations valid or not is up to you.

calambrac 17y ago

Twisted has always been an "emperor has no clothes" project to me, a project that's no doubt very useful but at the same time seems to revel in its unnecessary complexity (just read its documentation - how many times do you really have to tell us that something might seem overly complicated but is worth it for its flexibility?) I'm honestly glad to see a no-bullshit alternative that doesn't have that same sense of worshipping complexity as a sign of quality.

enneff 17y ago
That viewpoint seems quite narrow to me. The documentation of Twisted attempts to introduce several event-driven programming patterns. Most people have a 'wtf' moment or two when they come across design patterns that approach things seemingly backwards. The further I got into my Twisted app, the more I found myself reaching for things and finding that were already there for me. It was a great experience. You should try it!
- calambrac 17y ago
  I've done plenty of work with Twisted. My complaint was not that it lacked functionality, my complaint was that the culture of the project favors 'flexibility' over everything else. It's like the C++ of networking libraries. I'm sure there are plenty of times and places where its approach is exactly what you need, but it's just rarely a good fit for what I need.

Tichy 17y ago

It sounds very good, but I must admit I immediately remembered lots and lots of comments from open source developers that amounted to "not happy with our project (a ka 'free toilet')? well duh, hack it yourself, we owe you s*". Obviously not the fault of the Twisted developer, but I am not surprised if people actually prefer to hack their own stuff.

paul 17y ago

The great thing about open source is that people are free to share ideas and code. If the Twisted team likes parts of Tornado, they are free to replicate those parts back into Twisted while ignoring the parts that they don't like. Meanwhile, users are free to choose whichever approach suits them better. Twisted is no doubt more complete in terms of protocols and such, but many people also value simplicity. One important point that I think has been overlooked here is that the core of Tornado is only a few hundred lines of code. For me, the fact that I can actually read _all_ of the code in just a few minutes is very valuable. I'm sure many others don't care though, which is why it's great that we live in a world where there doesn't always have to be a single "right" answer.

dlsspy 17y ago
"Twisted is no doubt more complete in terms of protocols and such, but many people also value simplicity. One important point that I think has been overlooked here is that the core of Tornado is only a few hundred lines of code." I think this is a bit misleading. The twisted port involved removing almost 1,400 lines of code and didn't change the tornado API. That is, tornado is not more simple without twisted. I do understand the value of being able to read all of the code quickly, but there are other turtles below these. For me, it's very valuable to know that the part upon which I'm building is a well-tested codebase whose semantics I understand, and for which I can write tests when things begin to violate my understanding. If tornado is used in more applications, it will have to grow in complexity to meet the needs of those applications. I see tornado as having two faces. The web portions can certainly continue to grow, but the network IO handing just means that people will have to port over things that already exist within twisted, and expand it in complexity (such as the kqueue work that is in progress) to handle more low-level stuff. At some point, you can't just easily sit down and read all of it (at least, not any more easily than you can with twisted).
- paul 17y ago
  Tornado relies on a small number of well defined system calls (such as epoll). For me, the difference between reading (and debugging) a few hundred lines of code down to the system call level, and reading (and debugging) a big library like Twisted is rather significant. For you, the difference may not be important, and that's fine. I also don't think that it's necessary for Tornado to replicate all the functionality of Twisted -- if people want Twisted, they should simply use Twisted and ignore Tornado (and hopefully the Twisted team will have borrowed the best bits of Tornado anyway).

frognibble 17y ago

There are several async programming libraries for Python including Allegra ( http://laurentszyster.be/blog/allegra/doc/ http://laurentszyster.be/blog/allegra/doc/ ), libevent-python (and other libevent bindings to python), the standard asyncore module and more. Why should Facebook use Twisted instead of one of these other libraries?

imbaczek 17y ago
more importantly, why should it roll it's own given a plethora of more mature alternatives?
- frognibble 17y ago
  Perhaps there are many alternatives because they all have problems.