5 ms·

Also, don't forget that threads don't scale beyond one machine; so when you want expand beyond one machine you need to take care of two levels of scaling, threa

Also, don't forget that threads don't scale beyond one machine; so when you want expand beyond one machine you need to take care of two levels of scaling, threads and processes. This can be avoided when using multiple processes that communicate or share state some other way in the first place. Even with one CPU, with many cores the internal synchronization that happens to "simulate" a shared memory space can be expensive, if you're not careful your cores get clogged ping-ponging memory pages between each other. It might sound more convenient to use threads instead of processes, but in the end I'm not sure that all the work to remove the GIL (and introduce shitloads of finer-grained synchronization primitives) is worth it.

bad_user 15y ago
Threads are important not because you want to scale horizontally, but because you want to also scale vertically, as in having an optimum ratio of performance per Watts or CPU cores or RAM MB. Threads are much, much more efficient than processes - threads consume less memory (even with COW, VMs with garbage-collectors basically prevent COW from being effective), threads have faster context-switch, and threads achieve better cache-locality. And shared memory, which is considered the plague of the software industry along with pointers to memory, is actually a mirror of our current hardware architecture. The fact is that hard-disks and SSDs are much slower than RAM, RAM memory is much slower than L2 cache, which is slower than L1 cache, which is slower than when registers get accessed. You achieve good performance characteristics when you keep your data in one place, keeping your most accessed items in the CPU's cache, or at least prevent those items from being stored in swap. Food for thought: the NoSQL solutions so loved by Rails or Node.js developers are mostly written in C/C++ and rely on Posix threads, with some Erlang here and there. might sound more convenient to use threads instead of processes That's because it IS more convenient to use threads instead of processes in most cases. You can mostly get away with it only when the processes don't need to synchronize (i.e. your workload can be processed fully in parallel), but when processes do need to synchronize lots of bad shit can happen, as processes are more unreliable than threads. Of course, I'm referring to real kernel-managed POSIX threads and processes, not what Erlang does, but then again, the Erlang's light-weight processes are just an abstraction over POSIX threads, not processes as that would have been dumb.
- wladimir 15y ago
  Threads are much, much more efficient than processes - threads consume less memory (even with COW, VMs with garbage-collectors basically prevent COW from being effective), threads have faster context-switch, and threads achieve better cache-locality. There is hardly any "context switch" if you run one process per core (compared to one thread per core, it makes no difference). And yes, you'll save memory if you do everything in one process (even with COW), but don't forget garbage collection with multiple threads is a much more complex beast than single-threaded, resulting in extra overhead. I'm not denying that threads have some advantages, for example, when parallelizing heavy number-crunching work, but the point I was trying to make is that "they are much more efficient" is not that clear-cut and universally true. That's because it IS more convenient to use threads instead of processes in most cases Not necessarily true either. When using threads there is a much larger chance of introducing race conditions and deadlocks and other synchronization fails. It might look more convenient when writing the code, but debugging and supporting it certainly isn't.
  - buff-a 15y ago
    There is hardly any "context switch" if you run one process per core There is hardly any context switch if I don't run any software at all on my server. Zero bugs either. Back to the real world, and the point of this discussion, we run multiple Ruby processes per CPU so that when one process is waiting for the db or memcache, another can be doing useful work on the same CPU. When using threads there is a much larger chance of introducing race conditions and deadlocks and other synchronization fails As opposed to dropping the problem into memcache and getting it wrong: your code "works", you get no crashes, but customers lose data, lose posts, or lose money. If you're advocating using memcache because multithreading is too hard for your programmer, then your customers are fucked.
    - wladimir 15y ago
      We run multiple Ruby processes per CPU so that when one process is waiting for the db or memcache, another can be doing useful work on the same CPU That's the trivial case. When waiting for the db or memcache (I/O in general), the GIL is lifted, so it doesn't get in the way. It only gets in the way if multiple threads are inside the ruby VM doing work.

bad_user 15y ago

Threads are important not because you want to scale horizontally, but because you want to also scale vertically, as in having an optimum ratio of performance per Watts or CPU cores or RAM MB. Threads are much, much more efficient than processes - threads consume less memory (even with COW, VMs with garbage-collectors basically prevent COW from being effective), threads have faster context-switch, and threads achieve better cache-locality. And shared memory, which is considered the plague of the software industry along with pointers to memory, is actually a mirror of our current hardware architecture. The fact is that hard-disks and SSDs are much slower than RAM, RAM memory is much slower than L2 cache, which is slower than L1 cache, which is slower than when registers get accessed. You achieve good performance characteristics when you keep your data in one place, keeping your most accessed items in the CPU's cache, or at least prevent those items from being stored in swap. Food for thought: the NoSQL solutions so loved by Rails or Node.js developers are mostly written in C/C++ and rely on Posix threads, with some Erlang here and there. might sound more convenient to use threads instead of processes That's because it IS more convenient to use threads instead of processes in most cases. You can mostly get away with it only when the processes don't need to synchronize (i.e. your workload can be processed fully in parallel), but when processes do need to synchronize lots of bad shit can happen, as processes are more unreliable than threads. Of course, I'm referring to real kernel-managed POSIX threads and processes, not what Erlang does, but then again, the Erlang's light-weight processes are just an abstraction over POSIX threads, not processes as that would have been dumb.

wladimir 15y ago
Threads are much, much more efficient than processes - threads consume less memory (even with COW, VMs with garbage-collectors basically prevent COW from being effective), threads have faster context-switch, and threads achieve better cache-locality. There is hardly any "context switch" if you run one process per core (compared to one thread per core, it makes no difference). And yes, you'll save memory if you do everything in one process (even with COW), but don't forget garbage collection with multiple threads is a much more complex beast than single-threaded, resulting in extra overhead. I'm not denying that threads have some advantages, for example, when parallelizing heavy number-crunching work, but the point I was trying to make is that "they are much more efficient" is not that clear-cut and universally true. That's because it IS more convenient to use threads instead of processes in most cases Not necessarily true either. When using threads there is a much larger chance of introducing race conditions and deadlocks and other synchronization fails. It might look more convenient when writing the code, but debugging and supporting it certainly isn't.
- buff-a 15y ago
  There is hardly any "context switch" if you run one process per core There is hardly any context switch if I don't run any software at all on my server. Zero bugs either. Back to the real world, and the point of this discussion, we run multiple Ruby processes per CPU so that when one process is waiting for the db or memcache, another can be doing useful work on the same CPU. When using threads there is a much larger chance of introducing race conditions and deadlocks and other synchronization fails As opposed to dropping the problem into memcache and getting it wrong: your code "works", you get no crashes, but customers lose data, lose posts, or lose money. If you're advocating using memcache because multithreading is too hard for your programmer, then your customers are fucked.
  - wladimir 15y ago
    We run multiple Ruby processes per CPU so that when one process is waiting for the db or memcache, another can be doing useful work on the same CPU That's the trivial case. When waiting for the db or memcache (I/O in general), the GIL is lifted, so it doesn't get in the way. It only gets in the way if multiple threads are inside the ruby VM doing work.