4 ms·

Python in general doesn't really have a good solution for this, so it's not something specific to Django. I run a Python web app that has certain real-time need

Python in general doesn't really have a good solution for this, so it's not something specific to Django. I run a Python web app that has certain real-time needs, and I had to forgo a popular web framework like Django so that I could use Twisted. The problem with solutions like this is that since the language doesn't have built-in support for asynchronous IO, everything has to be compatible with the library of your choice (whether that's Twisted, Gevent, or other), and at that point, you'd be better off just using a different language/runtime like Node.js or Erlang. I think the current solution is to have Django serve the main app and have a separate "API server" that runs Node or whatever, but as the article points out, you're not really even using Django at that point because all it's doing is serving up a single HTML page--the rest is handled by the browser and the API server.

adamauckland 13y ago
That's currently what I do. Also Django is useful for sketching your requirements out, then once you know what you're building you can move the real-time data retrieval into NodeJS and let Django handle the boring stuff. Not EVERYTHING in an app will need to be real-time. Especially boring maintenance functionality such as password reset etc.
- RussianCow 13y ago
  True, but for such basic things, you could just use something lightweight like Flask and not have to deal with all of Django's baggage. It all depends on the app, though. In my case, even "boring" stuff often needed to make HTTP requests to the outside world, so Django would never have scaled for me. (In hindsight, I should have just written the whole thing in Node, but it wasn't as mature back then and I didn't want to take the risk.)
nostrademons 13y ago
Python 3.4 may help a lot with the language mechanisms (with asyncio, pluggable event loops, and composable generators everywhere), but there's still the issue of getting library support to use all of that. Node.js isn't actually better - it uses the callback model of async programming, which should be familiar to any C++ programmer who's been writing servers since the 80s, both because it's the current best solution for writing scalable event-driven servers and because it sucks. For ease of programming a CSP-based language like Go or Erlang is really the way to go, but then you're back to the "lack of library support" problem that you'd get with Python 3.4, except worse because Python at least has libraries for the synchronous part of the computation.
- steveklabnik 13y ago
  > Node.js isn't actually better - it uses the callback model of async programming, The point is not about the code style, it's that since Node is 'asynchronous by default,' culturally, Node produces libraries that are also asynchronous by default. Most Ruby or Python libraries aren't.
- Offler 13y ago
  Good news as you can now use ES6 generators in Node.js and when combined with another ES6 feature, Promises it can result in much nicer async code e.g. http://taskjs.org/ http://taskjs.org/ ES6 is looking very nice indeed, great thing is that with Node you won't have to worry about old browser support.
  - est 13y ago
    I never understand the `function *(){}` syntax. Why can't we simply introduce a new keyword like `generator (){}`?
    - netghost 13y ago
      Or just recognize that the function contains the yield keyword. That said, `function*` isn't all that bad given what you get in return.
- awj 13y ago
  > but there's still the issue of getting library support to use all of that. Much of Node's success in employing the event loop model comes from not having extensive libraries of blocking I/O code to work around. Maybe the difficulty of the 2-to-3 Python migration could be a boon here.
perlgeek 13y ago
Perl has libraries that can work with lots of different event loops (see for example http://search.cpan.org/~mlehmann/AnyEvent-7.07/lib/AnyEvent.pm#SUPPORTED_EVENT_LOOPS/BACKENDS http://search.cpan.org/~mlehmann/AnyEvent-7.07/lib/AnyEvent.... ) so asnyc IO doesn't need to be built in to support a very good level of portability. I'm sure that'd be possible in python as well.
- RussianCow 13y ago
  I guess it's less about native support and more about the culture. Very few Python libraries are built with async IO in mind. And honestly, I don't expect them to be, because there is no standard, agreed-upon way of doing async stuff in Python. So the result is you get a bunch of tiny communities around each event loop implementation, but because they aren't compatible, you get a bunch of repeated work and very little overall progress. (Don't know if it's the same with Perl, but I feel like the Perl community evolves much more quickly than Python's.)

adamauckland 13y ago

That's currently what I do. Also Django is useful for sketching your requirements out, then once you know what you're building you can move the real-time data retrieval into NodeJS and let Django handle the boring stuff. Not EVERYTHING in an app will need to be real-time. Especially boring maintenance functionality such as password reset etc.

RussianCow 13y ago
True, but for such basic things, you could just use something lightweight like Flask and not have to deal with all of Django's baggage. It all depends on the app, though. In my case, even "boring" stuff often needed to make HTTP requests to the outside world, so Django would never have scaled for me. (In hindsight, I should have just written the whole thing in Node, but it wasn't as mature back then and I didn't want to take the risk.)

nostrademons 13y ago

Python 3.4 may help a lot with the language mechanisms (with asyncio, pluggable event loops, and composable generators everywhere), but there's still the issue of getting library support to use all of that. Node.js isn't actually better - it uses the callback model of async programming, which should be familiar to any C++ programmer who's been writing servers since the 80s, both because it's the current best solution for writing scalable event-driven servers and because it sucks. For ease of programming a CSP-based language like Go or Erlang is really the way to go, but then you're back to the "lack of library support" problem that you'd get with Python 3.4, except worse because Python at least has libraries for the synchronous part of the computation.

steveklabnik 13y ago
> Node.js isn't actually better - it uses the callback model of async programming, The point is not about the code style, it's that since Node is 'asynchronous by default,' culturally, Node produces libraries that are also asynchronous by default. Most Ruby or Python libraries aren't.
Offler 13y ago
Good news as you can now use ES6 generators in Node.js and when combined with another ES6 feature, Promises it can result in much nicer async code e.g. http://taskjs.org/ http://taskjs.org/ ES6 is looking very nice indeed, great thing is that with Node you won't have to worry about old browser support.
- est 13y ago
  I never understand the `function *(){}` syntax. Why can't we simply introduce a new keyword like `generator (){}`?
  - netghost 13y ago
    Or just recognize that the function contains the yield keyword. That said, `function*` isn't all that bad given what you get in return.
awj 13y ago
> but there's still the issue of getting library support to use all of that. Much of Node's success in employing the event loop model comes from not having extensive libraries of blocking I/O code to work around. Maybe the difficulty of the 2-to-3 Python migration could be a boon here.

perlgeek 13y ago

Perl has libraries that can work with lots of different event loops (see for example http://search.cpan.org/~mlehmann/AnyEvent-7.07/lib/AnyEvent.pm#SUPPORTED_EVENT_LOOPS/BACKENDS http://search.cpan.org/~mlehmann/AnyEvent-7.07/lib/AnyEvent.... ) so asnyc IO doesn't need to be built in to support a very good level of portability. I'm sure that'd be possible in python as well.

RussianCow 13y ago
I guess it's less about native support and more about the culture. Very few Python libraries are built with async IO in mind. And honestly, I don't expect them to be, because there is no standard, agreed-upon way of doing async stuff in Python. So the result is you get a bunch of tiny communities around each event loop implementation, but because they aren't compatible, you get a bunch of repeated work and very little overall progress. (Don't know if it's the same with Perl, but I feel like the Perl community evolves much more quickly than Python's.)