9 ms·

Greenlet-based event I/O Framework for Python

63 points by areski 12y ago

joshbaptiste 12y ago
PyOhio 2013 presentation http://pyvideo.org/video/2278/diesel-simple-and-scalable-network-applications http://pyvideo.org/video/2278/diesel-simple-and-scalable-net...
vhost- 12y ago
No Python3? I can't really find anything on supported versions. EDIT: Looks like it's only for >=2.6 https://github.com/jamwt/diesel/blob/master/setup.py#L3 https://github.com/jamwt/diesel/blob/master/setup.py#L3 That's unfortunate because I need a clean event framework for Python3.
- natmaster 12y ago
  gevent should have py3 support soonish since dropbox put their money behind it: https://github.com/surfly/gevent/issues/38 https://github.com/surfly/gevent/issues/38 You can use gevent in a similar manner to this - monkeypatching is not required
- techdragon 12y ago
  asyncio - 3.4 has event driven built in goodness trollius - a backport of the 3.4 asyncio library for python 2.7 also take a look at Pulsar its an event driven / Actor model concurrency framework that supports 2.7 and up using asyncio and trollius
mcescalante 12y ago
I'd be curious to see this benchmarked against Tornado, Twisted, gevent w/ Flask, and Node.js. Would be fun to see where it fit into the possibilities for the popular frameworks for writing realtime web stuff. I'm starting to write a realtime backend right now in Golang and am excited to see how it goes. Really enjoying concurrency with Go so far. Would be glad to hear anybodies opinions that have done any realtime backends in Go :)
- joshbaptiste 12y ago
  This was a famous but now dated (2009) socket benchmark of frameworks that involved Diesel http://nichol.as/asynchronous-servers-in-python http://nichol.as/asynchronous-servers-in-python
  - rdtsc 12y ago
    I wouldn't look at that. That is 5 years old, and that was a different Diesel framework. Ok, same name (a cool one, I must admit), and I think same group of people. But, it did concurrency differently. It used to be a yield based API. Which, funny enough, is like Guido's latest Tulip/Async.IO approach. And it seems me they (Diesel) decided against it, and I 100% agree. Having yields in the library code, aside from demos and small examples, doesn't work very well. I think Diesel creators came to their approach from a practical standpoint. And I think Guido (and other proponents of async yield based IO) came from a purist/theoretical approach. We'll see what will happen in the future. It is hard to tell. But I don't see yields everywhere as a good thing. Just like I don't see callbacks or deferreds everywhere as a good way to do concurrency. greenlet (the base module gevent and eventlet are based on it) is probably the best thing that happened to Python in its recent history. It is really unfortunate it wasn't adopted as the main concurrency mechanism in the latest version.
    - fzzzy 12y ago
      "yield from" isn't available on Python 2.7, maybe that's why they switched?
      - jamwt 12y ago
        That would have helped a lot, yeah. Also, dealing with exceptions in an intuitive way when you have a "fake" stack you're managing yourself is a drag. The performance of of managing your own stack also sucks. But probably the biggest practical issue is there is a terrible, common silent mistake pattern, that bit us over and over again: self.do_something_async() # it's a generator What you really wanted to do was to `yield` this so the reactorish element in the frame above you consumes it. Instead, you created a NOOP generator that is never consumed, so whatever work you thought you did, silently.. does nothing. Anyway, I think `yield from` helps you get around some of this stuff, but I haven't been keeping up with tulip. Which is probably kind of silly since Guido works like 50 feet away from me. Btw, hi Donovan. :-)
girvo 12y ago
A little bit meta, but: The funniest part of sites like these with real-time chats getting posted, is seeing how prim and proper our comments on HN are compared to what we're all like in a chat box.
- jkarneges 12y ago
  Let's hope this never gets popular: http://hnlivecomments.pex2.jp/ http://hnlivecomments.pex2.jp/
- droope 12y ago
  a trip into the past :P
- Twisol 12y ago
  There's something adorably quaint about it. Multiple people can use the same name (and somebody's exploiting that to great success right now), but stuff like this still happens: <CSS DB>I'm waiting for HTTP 2 <111>why? <CSS DB>Just waiting <111>Let's wait together. <CSS DB>sounds nice .. 2 dude <111>after sucking such shit together, I see you all as a family <CSS DB>thank you for being that deep <CSS DB>I will save you wait for me where Adele is rolling <111>somebody has to Maybe I'm jaded, but this doesn't seem to happen in places where you have to worry about your reputation. And maybe you lose something by that. Or maybe I just don't get out enough.
  - rdtsc 12y ago
    I can imagined a funny scenario, where someone records and links it to the actual names or maybe just HN handles. Then one day (maybe tomorrow) reveals those links. Things like "HN handle Twisol said -- 'after sucking such shit together, I see you all as a family'. And then people look and see your other posts about serious topics as branch prediction in compilers and monetization strategies in start-ups.
vbit 12y ago
This is old. Is it still maintained after Google acquired Bump? Not much recent activity: https://github.com/jamwt/diesel/graphs/contributors https://github.com/jamwt/diesel/graphs/contributors
- jamwt 12y ago
  Dowski still works with it routinely at GOOG. I'm knee deep in golang at Dropbox myself these days, so I don't get around to working with diesel much.
notastartup 12y ago
this is good but I'm having trouble understanding the code on the first page. also some sort of benchmark would be very interesting. we desperately need something like meteor.js in python. imagine if you could write python on client and server and end up with something like meteor.js app

63 points by areski 12y ago

PyOhio 2013 presentation http://pyvideo.org/video/2278/diesel-simple-and-scalable-network-applications http://pyvideo.org/video/2278/diesel-simple-and-scalable-net...

vhost- 12y ago

No Python3? I can't really find anything on supported versions. EDIT: Looks like it's only for >=2.6 https://github.com/jamwt/diesel/blob/master/setup.py#L3 https://github.com/jamwt/diesel/blob/master/setup.py#L3 That's unfortunate because I need a clean event framework for Python3.

natmaster 12y ago
gevent should have py3 support soonish since dropbox put their money behind it: https://github.com/surfly/gevent/issues/38 https://github.com/surfly/gevent/issues/38 You can use gevent in a similar manner to this - monkeypatching is not required
techdragon 12y ago
asyncio - 3.4 has event driven built in goodness trollius - a backport of the 3.4 asyncio library for python 2.7 also take a look at Pulsar its an event driven / Actor model concurrency framework that supports 2.7 and up using asyncio and trollius

mcescalante 12y ago

I'd be curious to see this benchmarked against Tornado, Twisted, gevent w/ Flask, and Node.js. Would be fun to see where it fit into the possibilities for the popular frameworks for writing realtime web stuff. I'm starting to write a realtime backend right now in Golang and am excited to see how it goes. Really enjoying concurrency with Go so far. Would be glad to hear anybodies opinions that have done any realtime backends in Go :)

joshbaptiste 12y ago
This was a famous but now dated (2009) socket benchmark of frameworks that involved Diesel http://nichol.as/asynchronous-servers-in-python http://nichol.as/asynchronous-servers-in-python
- rdtsc 12y ago
  I wouldn't look at that. That is 5 years old, and that was a different Diesel framework. Ok, same name (a cool one, I must admit), and I think same group of people. But, it did concurrency differently. It used to be a yield based API. Which, funny enough, is like Guido's latest Tulip/Async.IO approach. And it seems me they (Diesel) decided against it, and I 100% agree. Having yields in the library code, aside from demos and small examples, doesn't work very well. I think Diesel creators came to their approach from a practical standpoint. And I think Guido (and other proponents of async yield based IO) came from a purist/theoretical approach. We'll see what will happen in the future. It is hard to tell. But I don't see yields everywhere as a good thing. Just like I don't see callbacks or deferreds everywhere as a good way to do concurrency. greenlet (the base module gevent and eventlet are based on it) is probably the best thing that happened to Python in its recent history. It is really unfortunate it wasn't adopted as the main concurrency mechanism in the latest version.
  - fzzzy 12y ago
    "yield from" isn't available on Python 2.7, maybe that's why they switched?
    - jamwt 12y ago
      That would have helped a lot, yeah. Also, dealing with exceptions in an intuitive way when you have a "fake" stack you're managing yourself is a drag. The performance of of managing your own stack also sucks. But probably the biggest practical issue is there is a terrible, common silent mistake pattern, that bit us over and over again: self.do_something_async() # it's a generator What you really wanted to do was to `yield` this so the reactorish element in the frame above you consumes it. Instead, you created a NOOP generator that is never consumed, so whatever work you thought you did, silently.. does nothing. Anyway, I think `yield from` helps you get around some of this stuff, but I haven't been keeping up with tulip. Which is probably kind of silly since Guido works like 50 feet away from me. Btw, hi Donovan. :-)

girvo 12y ago

A little bit meta, but: The funniest part of sites like these with real-time chats getting posted, is seeing how prim and proper our comments on HN are compared to what we're all like in a chat box.

jkarneges 12y ago
Let's hope this never gets popular: http://hnlivecomments.pex2.jp/ http://hnlivecomments.pex2.jp/
droope 12y ago
a trip into the past :P
Twisol 12y ago
There's something adorably quaint about it. Multiple people can use the same name (and somebody's exploiting that to great success right now), but stuff like this still happens: <CSS DB>I'm waiting for HTTP 2 <111>why? <CSS DB>Just waiting <111>Let's wait together. <CSS DB>sounds nice .. 2 dude <111>after sucking such shit together, I see you all as a family <CSS DB>thank you for being that deep <CSS DB>I will save you wait for me where Adele is rolling <111>somebody has to Maybe I'm jaded, but this doesn't seem to happen in places where you have to worry about your reputation. And maybe you lose something by that. Or maybe I just don't get out enough.
- rdtsc 12y ago
  I can imagined a funny scenario, where someone records and links it to the actual names or maybe just HN handles. Then one day (maybe tomorrow) reveals those links. Things like "HN handle Twisol said -- 'after sucking such shit together, I see you all as a family'. And then people look and see your other posts about serious topics as branch prediction in compilers and monetization strategies in start-ups.

vbit 12y ago

This is old. Is it still maintained after Google acquired Bump? Not much recent activity: https://github.com/jamwt/diesel/graphs/contributors https://github.com/jamwt/diesel/graphs/contributors

jamwt 12y ago
Dowski still works with it routinely at GOOG. I'm knee deep in golang at Dropbox myself these days, so I don't get around to working with diesel much.

notastartup 12y ago

this is good but I'm having trouble understanding the code on the first page. also some sort of benchmark would be very interesting. we desperately need something like meteor.js in python. imagine if you could write python on client and server and end up with something like meteor.js app