3 ms·

The problem with Twisted is that everything needs to be rewritten for it, like a special snowflake. Hendrix is an example of that. The advantage of Django is yo

The problem with Twisted is that everything needs to be rewritten for it, like a special snowflake. Hendrix is an example of that. The advantage of Django is you can use any python library without issues.

jMyles 11y ago
Really? I have exactly the opposite impression: that Twisted is happy to consume any streams of data you provide, while Django is only happy if you comply with the dictates of the LazySettings object. In any case, I think you'll find hendrix unbelievably easy to use - here's a talk that gives some detail: https://www.youtube.com/watch?v=92VeMkjM1TQ https://www.youtube.com/watch?v=92VeMkjM1TQ
- ubernostrum 11y ago
  I think you may be talking past each other. If 100% of integration with the system consists of sending network requests to it and receiving network responses from it, then sure, it's not going to affect you much. But when your integration involves taking existing applications/libraries and having them share memory space and call into and be called by functions in Twisted, then you're looking at a different story. Asynchronous code is infectious -- unless everything in your codebase is written to be or play nice with async, you get no benefits from the async parts and in fact can often get a net harm as the async and non-async bits clash with each other.
  - retrogradeorbit 11y ago
    This is not about async in general. This is about twisted's approach to async. For instance, there are numerous other languages that handle async elegantly and all the libraries work and there are no special snowflakes. Go comes to mind. Asyncio in python is a better answer for Python. But twisted is not asyncio. Twisted remains it's own separate thing.
    - jMyles 11y ago
      > Asyncio in python is a better answer for Python. But twisted is not asyncio. To my way of thinking about these things, this is plainly a non-sequitur. Twisted and asyncio don't do, or even try to do, the same things. The only reason that they are comparable is because, for years, Python lacked adequate primitives for expressing concurrency. Thus, Twisted, and its event loop (the reactor), handle this. And it's true that the reactor may eventually be replaced by async IO. However, the core of what Twisted does (handle network (and other I/O) traffic in an elegant, adaptive, concurrent way) is not something that asyncio is designed to replicate.
- retrogradeorbit 11y ago
  Yes, really. What if the library you are using uses blocking IO? What if it reads a file using a plain 'with open("file") as fh:'? Now you've blocked every connection in your twisted server. No warnings or errors, just strange intermittent pauses on every connection leaving you scratching your head. This isn't weird esoteric packages either. Ever used requests? SQLAlchemy? Django's ORM? You can't use them with twisted. Twisted proponents don't openly say this up front as it's not a great story, and their response is usually "just" port the library. Or maybe, put it like this. If Django and Twisted work well together, why does Hendrix exist at all? Just use Django. Or why does klein exist? Just use werkzeug. You'll see lots of this rewriting snowflakes. Just a quick search gives me txamqp, txmongo, txmandrill, txflask, txcelery, txdocker, txldap, txgithub, txrequests, txstripe, txzookeeper and on and on and on. Have a look: https://pypi.python.org/pypi?%3Aaction=search&term=twisted&submit=search https://pypi.python.org/pypi?%3Aaction=search&term=twisted&s... And then there's multiple ones of each to choose from. Look at amqp or reddis. There's something like four implementations of each. Even something like requests, theres treq, rxrequests, requests-twisted. Which one should I use? And then there's Python 3. Only half of twisted works on Python 3. Which half? I guess one finds out when they give it a go.
  - narsil 11y ago
    This is a great summary. I considered Twisted prior to running into some of the issues you've described, and the complexity of the framework. I ended up using gevent and have been happy its monkey-patching of the socket and related libraries.
  - jMyles 11y ago
    > This isn't weird esoteric packages either. Ever used requests? SQLAlchemy? Django's ORM? You can't use them with twisted. I'm left completely scratching my head here. I use all three of these packages routinely with Twisted. Are you saying that you are trying to share the reactor thread while blocking with, for example, requests.get()? Obviously you can't do that - why would you even try? Just run requests.get() on another thread. On the other hand, if you, for example, use deferToThreadPool with a bunch of Django ORM calls, you can happily go on serving requests from another ThreadPool. This works like gravy. These various adapters that you've described: in every case, their existence is not a testament to an inability to run their respective libraries' blocking I/O with Twisted; on the contrary, they provide support for actually using them with a proper Twisted protocol. The fact that so many of them exist doesn't mean that Twisted is doing anything wrong, it means that people really like Twisted and want these various packages to be able to talk directly to a Protocol or Producer without having to maintain that code in their project. If you want to run Django in Twisted without hendrix, go ahead: use the WSGIServer built in Twisted. However, you'll probably find that you are building lots of tooling around the awesome things you want to do with Twisted (ThreadPool tuning, websockets, concurrent functions running in views, and much more). So, instead of having to write all that yourself, hendrix ships with it. For my practice, the Twisted ecosystem is the finest part of Python.
- techdragon 11y ago
  The "twisted problem" is often summarised as "I know Twisted is great for asynchronous Python... If only it was the kind of Python I want to write" It's not that Twisted is bad. It's just that many Python developers are not "fluent in the Twisted dialect" like how some people find a thick Scottish accent hard to understand and other more familiar with it can understand every word.

jMyles 11y ago

Really? I have exactly the opposite impression: that Twisted is happy to consume any streams of data you provide, while Django is only happy if you comply with the dictates of the LazySettings object. In any case, I think you'll find hendrix unbelievably easy to use - here's a talk that gives some detail: https://www.youtube.com/watch?v=92VeMkjM1TQ https://www.youtube.com/watch?v=92VeMkjM1TQ

ubernostrum 11y ago
I think you may be talking past each other. If 100% of integration with the system consists of sending network requests to it and receiving network responses from it, then sure, it's not going to affect you much. But when your integration involves taking existing applications/libraries and having them share memory space and call into and be called by functions in Twisted, then you're looking at a different story. Asynchronous code is infectious -- unless everything in your codebase is written to be or play nice with async, you get no benefits from the async parts and in fact can often get a net harm as the async and non-async bits clash with each other.
- retrogradeorbit 11y ago
  This is not about async in general. This is about twisted's approach to async. For instance, there are numerous other languages that handle async elegantly and all the libraries work and there are no special snowflakes. Go comes to mind. Asyncio in python is a better answer for Python. But twisted is not asyncio. Twisted remains it's own separate thing.
  - jMyles 11y ago
    > Asyncio in python is a better answer for Python. But twisted is not asyncio. To my way of thinking about these things, this is plainly a non-sequitur. Twisted and asyncio don't do, or even try to do, the same things. The only reason that they are comparable is because, for years, Python lacked adequate primitives for expressing concurrency. Thus, Twisted, and its event loop (the reactor), handle this. And it's true that the reactor may eventually be replaced by async IO. However, the core of what Twisted does (handle network (and other I/O) traffic in an elegant, adaptive, concurrent way) is not something that asyncio is designed to replicate.
retrogradeorbit 11y ago
Yes, really. What if the library you are using uses blocking IO? What if it reads a file using a plain 'with open("file") as fh:'? Now you've blocked every connection in your twisted server. No warnings or errors, just strange intermittent pauses on every connection leaving you scratching your head. This isn't weird esoteric packages either. Ever used requests? SQLAlchemy? Django's ORM? You can't use them with twisted. Twisted proponents don't openly say this up front as it's not a great story, and their response is usually "just" port the library. Or maybe, put it like this. If Django and Twisted work well together, why does Hendrix exist at all? Just use Django. Or why does klein exist? Just use werkzeug. You'll see lots of this rewriting snowflakes. Just a quick search gives me txamqp, txmongo, txmandrill, txflask, txcelery, txdocker, txldap, txgithub, txrequests, txstripe, txzookeeper and on and on and on. Have a look: https://pypi.python.org/pypi?%3Aaction=search&term=twisted&submit=search https://pypi.python.org/pypi?%3Aaction=search&term=twisted&s... And then there's multiple ones of each to choose from. Look at amqp or reddis. There's something like four implementations of each. Even something like requests, theres treq, rxrequests, requests-twisted. Which one should I use? And then there's Python 3. Only half of twisted works on Python 3. Which half? I guess one finds out when they give it a go.
- narsil 11y ago
  This is a great summary. I considered Twisted prior to running into some of the issues you've described, and the complexity of the framework. I ended up using gevent and have been happy its monkey-patching of the socket and related libraries.
- jMyles 11y ago
  > This isn't weird esoteric packages either. Ever used requests? SQLAlchemy? Django's ORM? You can't use them with twisted. I'm left completely scratching my head here. I use all three of these packages routinely with Twisted. Are you saying that you are trying to share the reactor thread while blocking with, for example, requests.get()? Obviously you can't do that - why would you even try? Just run requests.get() on another thread. On the other hand, if you, for example, use deferToThreadPool with a bunch of Django ORM calls, you can happily go on serving requests from another ThreadPool. This works like gravy. These various adapters that you've described: in every case, their existence is not a testament to an inability to run their respective libraries' blocking I/O with Twisted; on the contrary, they provide support for actually using them with a proper Twisted protocol. The fact that so many of them exist doesn't mean that Twisted is doing anything wrong, it means that people really like Twisted and want these various packages to be able to talk directly to a Protocol or Producer without having to maintain that code in their project. If you want to run Django in Twisted without hendrix, go ahead: use the WSGIServer built in Twisted. However, you'll probably find that you are building lots of tooling around the awesome things you want to do with Twisted (ThreadPool tuning, websockets, concurrent functions running in views, and much more). So, instead of having to write all that yourself, hendrix ships with it. For my practice, the Twisted ecosystem is the finest part of Python.
techdragon 11y ago
The "twisted problem" is often summarised as "I know Twisted is great for asynchronous Python... If only it was the kind of Python I want to write" It's not that Twisted is bad. It's just that many Python developers are not "fluent in the Twisted dialect" like how some people find a thick Scottish accent hard to understand and other more familiar with it can understand every word.