4 ms·

All these systems are share-everything, which I'm not excited about -- if I want share-everything I can just use threads, and they work far better than most peo

by ianb 16y ago

All these systems are share-everything, which I'm not excited about -- if I want share-everything I can just use threads, and they work far better than most people seem willing to admit.

mrshoe 16y ago
Well threads certainly don't work well for high concurrency, which I read as your biggest impetus for a new runtime. I suppose if you mean true (multi-core) concurrency then you're right that Python has some big limitations, but that pretty much boils down to the age old "remove the GIL" proposal.
- ianb 16y ago
  Threads work pretty darn well, actually. Depending on the libraries you use the GIL may not get in the way (e.g., database libraries, stuff like lxml that releases the GIL). If you have blocking stuff like I/O (socket or file) then threads help. The only time when the GIL really gets in the way is with multiple processors and CPU-bound routines. Even with CPU-bound routines you are often better off with threads as the machinery does share the CPU transparently and relatively fairly. Threads are a little half-assed, sure, but so are most of the currently offered "replacements" for threads, including cooperative multitasking (which also doesn't do anything with multiple processors!)
  - mrshoe 16y ago
    I guess that depends on whether you define "high concurrency" as ~50 or more like ~5000. I'm guessing your "half-assed" comment is in reference to some of the sames things David Beazley mentioned in his PyCon talk, but it's worth linking to anyway: http://www.dabeaz.com/GIL/ http://www.dabeaz.com/GIL/ Basically threads work pretty darn well until you throw some real load at them.
    - ianb 16y ago
      Well, practice shows threads work pretty darn well with real load, as there are a huge number of running examples of high-load threaded Python applications. There is the potential of failure due to race conditions, but IMHO it's not as bad as it's often made out to be. For high concurrency, there is of course the question of what level in the stack you want to handle the concurrency at. There's not much use to doing 5000 things at the same time. But you might want to handle 5000 things at the same time (say, open sockets). The question is how much you want to twist your code around to do this -- with threads you might have worker pools, and you might have to be sure your chunks of work are of reasonable size (and often it's tricky to do some things incrementally). With cooperative concurrency... well, you do all the same things ;) But sometimes when threads are enough you don't have to contort your code, and in some error cases threads will help a lot (like you happen to not partition your work sufficiently). I really just want to get rid of those tweaky code things for all cases, so that it's not a matter of these different choices with different tradeoffs, but just one really good choice that applies much more widely.
      - mattbillenstein 16y ago
        Threads for python webapps are fine, but they break down when doing things like chat (using long polling or websockets) where you have a lot of open mostly idle connections. Coroutines and libraries like gevent and eventlet let you elegantly use greenlets without contorting your code -- ie, you get async without callbacks or deferreds -- and you have the benefit of being able to handle thousands of connections in a single python process. Of course you don't get processor concurrency, but you don't really get that with straight CPython because of the GIL anyway -- and if you're deploying a largish web app, you're going to have a load balancer (haproxy ftw) and you can just run one process per core anyway. This also mostly transparently lets you scale out to N boxes with M cores per box with very little architecture changes at the inbound request to web-server level. Which brings me to the microprocess / shared module ideas -- RAM is pretty cheap -- 1GB or 2GB per core isn't anything fancy, and if you're using more than a couple gig in a python webapp process, something is probably wrong. As far as sharing state between microprocesses, I don't see how that really solves anything -- you're going to have processes on different boxes if you have any sort of traffic or fault tolerance and you're going to be putting that shared state into a cache or database somewhere anyway. Raw compute speed is important to the scipy/numpy crowds, so I think things like Cython and PyPy make a lot of sense there -- webapps are I/O bound, so you spend 90% or more of the time it takes to service a request waiting, which async is really good at. PyPy isn't going to beat CPython by 20x on some django benchmark, but they will on some compute-intensive ones. So anyway, I think my point is, you can have your cake and eat it too -- coroutines without pain (gevent) and processor-level concurrency (load balancer) for web applications using off the shelf production-ready technology on commodity hardware...
- deleted 16y ago
  [deleted]
otterley 16y ago
Threads have at least the following problems in high concurrency environments: (1) context switching is expensive (2) access to shared variables leads to race conditions, which requires careful synchronization (to be fair, this is true for any shared storage: databases, mmap()ed regions, etc.) See also http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1...., which provides a thorough treatment on why multithreaded programs are inherently less reliable than single-threaded.
- mryall 16y ago
  > access to shared variables leads to race conditions ... Race conditions are a type of programmer error, not an invariable consequence of shared state. You seem to be implying that high concurrency increases the risk of race conditions, which it doesn't. A race condition is either present or it isn't. A correctly written application that uses threads isn't inherently less reliable than a single-threaded application, it is just harder to write.
- FixedPoint 16y ago
  Quoting Robert Harper: 'Concurrency is concerned with nondeterministic composition of programs (or their components).' [1] Threads can be a very efficient way of introducing non-determinism (preferable to incurring IPC overhead). Writing correct code then boils down to controlling interactions between threads. Actor architectures can be very helpful in such settings. Key to safe implementations are type-enforced isolation and immutability. More experimentally, STMs offer the promise of high-level composition of actors. My 2p. [1] http://existentialtype.wordpress.com/2011/03/17/parallelism-is-not-concurrency/ http://existentialtype.wordpress.com/2011/03/17/parallelism-...
jamwt 16y ago
Can you say more about what you mean when you said they're "share-everything?" I'm not entirely sure what you mean by that, since, for example, we do diesel in what we consider to be a "shared nothing" approach at Bump. Redis is the transport (queuing, pubsub) and the network-bound state, while diesel loops are stateless. In fact, the higher-level erlang-style stuff that's not released yet really abstracts the idea of queues and consumers to the point where it's not entirely clear or important if particular jobs are within the same process of even the same machine (in practice, they usually aren't).
- ianb 16y ago
  Generally cooperative multitasking systems like these (anything based on greenlets, I think) mean that you are running concurrent tasks in a single process, with a single set of shared data. It's basically the same as threads, only you can't have multiple tasks literally accessing the same data at the same time (though with the GIL I guess technically that's true with threads too ;).
  - euccastro 16y ago
    Just don't access the same data? Threads, greenlets, tasklets, etc. allow you to share state, but don't force you to. Greenlets/tasklets are game changing with respect to OS threads not because they introduce new ways of working, but because their performance and scalability afford you ways of working that would be theoretically possible but impractical with threads. The GIL only widens that gap.
  - jamwt 16y ago
    Well, it's only shared if you share it! I guess it boils down to "don't use mutable globals". That's as true in threaded environments as it is in async ones. If your requirement is that there must be system-enforced safety rather than safe practices and idioms, I'd say Python isn't the right tool (Haskell is) to begin with. As you know, Python is all about practices, idioms, and discipline.

by ianb 16y ago

All these systems are share-everything, which I'm not excited about -- if I want share-everything I can just use threads, and they work far better than most people seem willing to admit.

mrshoe 16y ago

Well threads certainly don't work well for high concurrency, which I read as your biggest impetus for a new runtime. I suppose if you mean true (multi-core) concurrency then you're right that Python has some big limitations, but that pretty much boils down to the age old "remove the GIL" proposal.

ianb 16y ago
Threads work pretty darn well, actually. Depending on the libraries you use the GIL may not get in the way (e.g., database libraries, stuff like lxml that releases the GIL). If you have blocking stuff like I/O (socket or file) then threads help. The only time when the GIL really gets in the way is with multiple processors and CPU-bound routines. Even with CPU-bound routines you are often better off with threads as the machinery does share the CPU transparently and relatively fairly. Threads are a little half-assed, sure, but so are most of the currently offered "replacements" for threads, including cooperative multitasking (which also doesn't do anything with multiple processors!)
- mrshoe 16y ago
  I guess that depends on whether you define "high concurrency" as ~50 or more like ~5000. I'm guessing your "half-assed" comment is in reference to some of the sames things David Beazley mentioned in his PyCon talk, but it's worth linking to anyway: http://www.dabeaz.com/GIL/ http://www.dabeaz.com/GIL/ Basically threads work pretty darn well until you throw some real load at them.
  - ianb 16y ago
    Well, practice shows threads work pretty darn well with real load, as there are a huge number of running examples of high-load threaded Python applications. There is the potential of failure due to race conditions, but IMHO it's not as bad as it's often made out to be. For high concurrency, there is of course the question of what level in the stack you want to handle the concurrency at. There's not much use to doing 5000 things at the same time. But you might want to handle 5000 things at the same time (say, open sockets). The question is how much you want to twist your code around to do this -- with threads you might have worker pools, and you might have to be sure your chunks of work are of reasonable size (and often it's tricky to do some things incrementally). With cooperative concurrency... well, you do all the same things ;) But sometimes when threads are enough you don't have to contort your code, and in some error cases threads will help a lot (like you happen to not partition your work sufficiently). I really just want to get rid of those tweaky code things for all cases, so that it's not a matter of these different choices with different tradeoffs, but just one really good choice that applies much more widely.
    - mattbillenstein 16y ago
      Threads for python webapps are fine, but they break down when doing things like chat (using long polling or websockets) where you have a lot of open mostly idle connections. Coroutines and libraries like gevent and eventlet let you elegantly use greenlets without contorting your code -- ie, you get async without callbacks or deferreds -- and you have the benefit of being able to handle thousands of connections in a single python process. Of course you don't get processor concurrency, but you don't really get that with straight CPython because of the GIL anyway -- and if you're deploying a largish web app, you're going to have a load balancer (haproxy ftw) and you can just run one process per core anyway. This also mostly transparently lets you scale out to N boxes with M cores per box with very little architecture changes at the inbound request to web-server level. Which brings me to the microprocess / shared module ideas -- RAM is pretty cheap -- 1GB or 2GB per core isn't anything fancy, and if you're using more than a couple gig in a python webapp process, something is probably wrong. As far as sharing state between microprocesses, I don't see how that really solves anything -- you're going to have processes on different boxes if you have any sort of traffic or fault tolerance and you're going to be putting that shared state into a cache or database somewhere anyway. Raw compute speed is important to the scipy/numpy crowds, so I think things like Cython and PyPy make a lot of sense there -- webapps are I/O bound, so you spend 90% or more of the time it takes to service a request waiting, which async is really good at. PyPy isn't going to beat CPython by 20x on some django benchmark, but they will on some compute-intensive ones. So anyway, I think my point is, you can have your cake and eat it too -- coroutines without pain (gevent) and processor-level concurrency (load balancer) for web applications using off the shelf production-ready technology on commodity hardware...
deleted 16y ago
[deleted]

otterley 16y ago

Threads have at least the following problems in high concurrency environments: (1) context switching is expensive (2) access to shared variables leads to race conditions, which requires careful synchronization (to be fair, this is true for any shared storage: databases, mmap()ed regions, etc.) See also http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1...., which provides a thorough treatment on why multithreaded programs are inherently less reliable than single-threaded.

mryall 16y ago
> access to shared variables leads to race conditions ... Race conditions are a type of programmer error, not an invariable consequence of shared state. You seem to be implying that high concurrency increases the risk of race conditions, which it doesn't. A race condition is either present or it isn't. A correctly written application that uses threads isn't inherently less reliable than a single-threaded application, it is just harder to write.
FixedPoint 16y ago
Quoting Robert Harper: 'Concurrency is concerned with nondeterministic composition of programs (or their components).' [1] Threads can be a very efficient way of introducing non-determinism (preferable to incurring IPC overhead). Writing correct code then boils down to controlling interactions between threads. Actor architectures can be very helpful in such settings. Key to safe implementations are type-enforced isolation and immutability. More experimentally, STMs offer the promise of high-level composition of actors. My 2p. [1] http://existentialtype.wordpress.com/2011/03/17/parallelism-is-not-concurrency/ http://existentialtype.wordpress.com/2011/03/17/parallelism-...

jamwt 16y ago

Can you say more about what you mean when you said they're "share-everything?" I'm not entirely sure what you mean by that, since, for example, we do diesel in what we consider to be a "shared nothing" approach at Bump. Redis is the transport (queuing, pubsub) and the network-bound state, while diesel loops are stateless. In fact, the higher-level erlang-style stuff that's not released yet really abstracts the idea of queues and consumers to the point where it's not entirely clear or important if particular jobs are within the same process of even the same machine (in practice, they usually aren't).

ianb 16y ago
Generally cooperative multitasking systems like these (anything based on greenlets, I think) mean that you are running concurrent tasks in a single process, with a single set of shared data. It's basically the same as threads, only you can't have multiple tasks literally accessing the same data at the same time (though with the GIL I guess technically that's true with threads too ;).
- euccastro 16y ago
  Just don't access the same data? Threads, greenlets, tasklets, etc. allow you to share state, but don't force you to. Greenlets/tasklets are game changing with respect to OS threads not because they introduce new ways of working, but because their performance and scalability afford you ways of working that would be theoretically possible but impractical with threads. The GIL only widens that gap.
- jamwt 16y ago
  Well, it's only shared if you share it! I guess it boils down to "don't use mutable globals". That's as true in threaded environments as it is in async ones. If your requirement is that there must be system-enforced safety rather than safe practices and idioms, I'd say Python isn't the right tool (Haskell is) to begin with. As you know, Python is all about practices, idioms, and discipline.