4 ms·

Or just use gevent or eventlet. It's not necessary to use a whole different branch of python. On top of that, gevent outperforms both twisted and tornado becau

by j2labs 15y ago

Or just use gevent or eventlet. It's not necessary to use a whole different branch of python. On top of that, gevent outperforms both twisted and tornado because the eventloop is libev, meaning it's written in C instead of pure python.

gauravk92 15y ago
Can you find a source for that, I'd love to read about actual performance statistics between anything and the likes ok twisted and such.
- jeangenie 15y ago
  http://teddziuba.com/2010/02/eventlet-asynchronous-io-for-g.html http://teddziuba.com/2010/02/eventlet-asynchronous-io-for-g....
  - j2labs 15y ago
    This link is flawed. It points towards eventlet but gevent is significantly better. Not sure why Ted felt otherwise, but I speculate that it's because at the time of writing gevent was considered incomplete compared to eventlet. To be clear: gevent is significantly faster than eventlet and offers the same functionality.
- j2labs 15y ago
  http://nichol.as/benchmark-of-python-web-servers http://nichol.as/benchmark-of-python-web-servers
- glyph 15y ago
  Twisted's event loop is faster than gevent and tornado. http://cyclone.io/ http://cyclone.io/
  - j2labs 15y ago
    This test is for a pointlessly weak concurrency level of 25. Under serious load the difference is much more gigantic showing that Twisted can't handle the same load as gevent.
- glyph 15y ago
  Twisted has a benchmark suite and release-to-release tracking of performance information, so that we can be sure optimizations have a positive impact. http://speed.twistedmatrix.com/timeline/ http://speed.twistedmatrix.com/timeline/
sausagefeet 15y ago
The main problem I see with gevent/eventlet is it has many of the weaknesses of callback and preemptive threaded code with none of the strengths. - It can't use multiple cores, even if Python got the ability in the future, gevent code simply wouldn't work as intended. - While gevent is not preemptive, context switches are implicit, that means you cannot write code without locks unless you want to do some handwaving. You have to know the entire system to know that a multistep operation that calls functions is actually safe. On the net, gevent may be better than Twisted or Tornado, but I think that is saying more about how much those suck than how good gevent is.
- j2labs 15y ago
  > The main problem I see with gevent/eventlet is it has many of the weaknesses of callback and preemptive threaded code with none of the strengths. It is intended to create an environment similar to preemptive threads, yet due to it being a networking library it is unlikely gevent users will have to use the semaphore and locking primitives provided with gevent. Regarding the weaknesses of callbacks, I disagree and would like to see evidence supporting your case. > - It can't use multiple cores, even if Python got the ability in the future, gevent code simply wouldn't work as intended. This is pure speculation. If Python's multicore handling was better (existed?), it's likely gevent would be implemented similar to other systems where you have threads allocated to each core and the concurrency primitives can then be executed on cores according to some algorithm. > - While gevent is not preemptive, context switches are implicit, that means you cannot write code without locks unless you want to do some handwaving. You have to know the entire system to know that a multistep operation that calls functions is actually safe. While this is true, I don't think it is likely to occur. Gevent's creator agrees: http://www.gevent.org/intro.html#cooperative-multitasking http://www.gevent.org/intro.html#cooperative-multitasking > On the net, gevent may be better than Twisted or Tornado, but I think that is saying more about how much those suck than how good gevent is. You're an Erlang programmer. You have to have that opinion. (sausagefeet and I are friends IRL so I know his biases)
  - sausagefeet 15y ago
    > Regarding the weaknesses of callbacks, I disagree and would like to see evidence supporting your case. The weakness of callbacks is the lack of multicore support. It is not pure speculation that gevent simply won't work in a preemptive environment if it wants to utilize multiple cores. The sentence right about this makes my point: > yet due to it being a networking library it is unlikely gevent users will have to use the semaphore and locking primitives provided with gevent. Why is that? It's because gevent context switches on I/O events, not preemptively, so you have some control over when context switches happen. You can depend on 'my_dict[foo] += 1' to happen atomically. If you want to utilize multiple cores with gevent, as code using it is written now, you can't because all of the code out there depend on operations between I/O events happening atomically. This isn't a secret in the callback/gevent world. On the contrary, it's presented as a selling point. No longer worry about those locking primitives those real threading libraries give you because we are single threaded. (Ocaml/Lwt has the same problem). > While this is true, I don't think it is likely to occur I'll concede it's far less likely. I'm curious how large gevent code bases are in the wild, a large application that one cannot fit in their entire head could very well have a lot of the same race conditions that a preemptively threaded application has. > You're an Erlang programmer. You have to have that opinion Really? I didn't realize when I started using Erlang I signed a contract saying that. I also write Ocaml, so I guess any criticism I levy against Java is a product of that too, not rational thought.
glyph 15y ago
Python outperforms C in many cases. http://morepypy.blogspot.com/2011/08/pypy-is-faster-than-c-again-string.html http://morepypy.blogspot.com/2011/08/pypy-is-faster-than-c-a...
- j2labs 15y ago
  Will be awesome to see more of what we get from pypy. Eventlet will probably gain a lot too, for those with a taste for the implicit eventloop.

by j2labs 15y ago

gauravk92 15y ago

Can you find a source for that, I'd love to read about actual performance statistics between anything and the likes ok twisted and such.

jeangenie 15y ago
http://teddziuba.com/2010/02/eventlet-asynchronous-io-for-g.html http://teddziuba.com/2010/02/eventlet-asynchronous-io-for-g....
- j2labs 15y ago
  This link is flawed. It points towards eventlet but gevent is significantly better. Not sure why Ted felt otherwise, but I speculate that it's because at the time of writing gevent was considered incomplete compared to eventlet. To be clear: gevent is significantly faster than eventlet and offers the same functionality.
j2labs 15y ago
http://nichol.as/benchmark-of-python-web-servers http://nichol.as/benchmark-of-python-web-servers
glyph 15y ago
Twisted's event loop is faster than gevent and tornado. http://cyclone.io/ http://cyclone.io/
- j2labs 15y ago
  This test is for a pointlessly weak concurrency level of 25. Under serious load the difference is much more gigantic showing that Twisted can't handle the same load as gevent.
glyph 15y ago
Twisted has a benchmark suite and release-to-release tracking of performance information, so that we can be sure optimizations have a positive impact. http://speed.twistedmatrix.com/timeline/ http://speed.twistedmatrix.com/timeline/

sausagefeet 15y ago

The main problem I see with gevent/eventlet is it has many of the weaknesses of callback and preemptive threaded code with none of the strengths. - It can't use multiple cores, even if Python got the ability in the future, gevent code simply wouldn't work as intended. - While gevent is not preemptive, context switches are implicit, that means you cannot write code without locks unless you want to do some handwaving. You have to know the entire system to know that a multistep operation that calls functions is actually safe. On the net, gevent may be better than Twisted or Tornado, but I think that is saying more about how much those suck than how good gevent is.

j2labs 15y ago
> The main problem I see with gevent/eventlet is it has many of the weaknesses of callback and preemptive threaded code with none of the strengths. It is intended to create an environment similar to preemptive threads, yet due to it being a networking library it is unlikely gevent users will have to use the semaphore and locking primitives provided with gevent. Regarding the weaknesses of callbacks, I disagree and would like to see evidence supporting your case. > - It can't use multiple cores, even if Python got the ability in the future, gevent code simply wouldn't work as intended. This is pure speculation. If Python's multicore handling was better (existed?), it's likely gevent would be implemented similar to other systems where you have threads allocated to each core and the concurrency primitives can then be executed on cores according to some algorithm. > - While gevent is not preemptive, context switches are implicit, that means you cannot write code without locks unless you want to do some handwaving. You have to know the entire system to know that a multistep operation that calls functions is actually safe. While this is true, I don't think it is likely to occur. Gevent's creator agrees: http://www.gevent.org/intro.html#cooperative-multitasking http://www.gevent.org/intro.html#cooperative-multitasking > On the net, gevent may be better than Twisted or Tornado, but I think that is saying more about how much those suck than how good gevent is. You're an Erlang programmer. You have to have that opinion. (sausagefeet and I are friends IRL so I know his biases)
- sausagefeet 15y ago
  > Regarding the weaknesses of callbacks, I disagree and would like to see evidence supporting your case. The weakness of callbacks is the lack of multicore support. It is not pure speculation that gevent simply won't work in a preemptive environment if it wants to utilize multiple cores. The sentence right about this makes my point: > yet due to it being a networking library it is unlikely gevent users will have to use the semaphore and locking primitives provided with gevent. Why is that? It's because gevent context switches on I/O events, not preemptively, so you have some control over when context switches happen. You can depend on 'my_dict[foo] += 1' to happen atomically. If you want to utilize multiple cores with gevent, as code using it is written now, you can't because all of the code out there depend on operations between I/O events happening atomically. This isn't a secret in the callback/gevent world. On the contrary, it's presented as a selling point. No longer worry about those locking primitives those real threading libraries give you because we are single threaded. (Ocaml/Lwt has the same problem). > While this is true, I don't think it is likely to occur I'll concede it's far less likely. I'm curious how large gevent code bases are in the wild, a large application that one cannot fit in their entire head could very well have a lot of the same race conditions that a preemptively threaded application has. > You're an Erlang programmer. You have to have that opinion Really? I didn't realize when I started using Erlang I signed a contract saying that. I also write Ocaml, so I guess any criticism I levy against Java is a product of that too, not rational thought.

glyph 15y ago

Python outperforms C in many cases. http://morepypy.blogspot.com/2011/08/pypy-is-faster-than-c-again-string.html http://morepypy.blogspot.com/2011/08/pypy-is-faster-than-c-a...

j2labs 15y ago
Will be awesome to see more of what we get from pypy. Eventlet will probably gain a lot too, for those with a taste for the implicit eventloop.