4 ms·

#1 & #2: Consumer CPUs are now pushing 16 cores & 32 threads. Python is limited to ~1/20th of what a single box is capable of. That's a pretty big bottleneck.

#1 & #2: Consumer CPUs are now pushing 16 cores & 32 threads. Python is limited to ~1/20th of what a single box is capable of. That's a pretty big bottleneck. #4: Even if you're just talking message passing sending a message between threads is in the 10s of nanoseconds while between processes is 10s of microseconds. That's a ~1000x slowdown on core communication. Given that CPU cores are not getting any faster, that's a pretty big hit to efficiency to take. Similarly simply moving data between processes is expensive, while moving data between threads is free.

sqeaky 9y ago
You are right about the majority of what you said, but I am pedantically picking on one point. CPU cores are getting faster, but they aren't doing it with clock speed, they are dispatching more instructions per cycle or otherwise making the work faster.
- kllrnohj 9y ago
  IPC gains per generation are vanishingly tiny, if they exist at all. Skylake -> Kaby Lake, for example, had no IPC improvements at all. A very small clock bump to the various tiers was it. Even if you look over a large generation gap there's only a ~20% IPC improvement going from an i7-2600K to an i7-7700K ( https://www.hardocp.com/article/2017/01/13/kaby_lake_7700k_vs_sandy_bridge_2600k_ipc_review/ https://www.hardocp.com/article/2017/01/13/kaby_lake_7700k_v... ) 6 years & a shrink from 32nm to 14nm and all it can muster is +20%. Cores are just not getting faster by any meaningful amount.
  - happycube 9y ago
    Yeah - I remember when five year old CPUs were basically useless! (Kaby Lake is basically a new stepping of Skylake - if intel wasn't having problems with new process nodes it likely wouldn't have been released at all, and if it was it would've been used for a one-off chip in the same generation ala the 4770K)
  - sqeaky 9y ago
    My compiles have gotten more than 20% faster so something is making my newere machines faster than my older machines. That it is not 10x as fast I blame on AMD for not being as competitive as they could have been.
jononor 9y ago
The efficiency hit is very dependent on how large your computation chunks are. If the computation per message batch is on order of 100 ms, it would be <10% loss.
- kllrnohj 9y ago
  Assuming very small messages that are rarely sent then yes, the hit of multi process is not going to be your biggest issue.
breatheoften 9y ago
Moving data between threads is only free to the extent that synchronization is free. Maybe you could say that moving immutable data between threads is free but I don't think you can say its free in general ... Doing so significantly undersells the complexity that comes with shared memory concurrency.
- kllrnohj 9y ago
  You seem to be conflating moving with sharing. Moving between threads is always free[1] regardless of if it's mutable or immutable, and there's no concurrency issues at all since it's a move. Move means the sender no longer has a reference. As in, std::move, rust's ownership transfer, webworker's transferables, etc... 1: Yes there's a single synchronize point where the handoff happens, but this is part of sending a message at all. It's also independent to the size & complexity of the payload itself when we're talking multi-threaded instead of multi-process. You have that exact same sync point that costs the exact same regardless of whether your message consists of a single byte or a multi-gigabyte structure.
  - breatheoften 9y ago
    Ah I see -- Can you actually describe ownership in python sufficiently well to be able to describe this move operation for any useful python data structures?
    - kllrnohj 9y ago
      Generically ownership is simply who has a reference to the object. So if a.foo = b, then a 'owns' b. A 'move' is simply handing a different object the reference, then dropping your own reference. For example: a.foo = b // a 'owns' b c.foo = a.foo // a & c share b, however if the next line is: a.foo = None // a has 'moved' b to c, since c now has the only reference to b. Some languages have codified this to make the contract part of the language, but it doesn't need any first-class language support. It's just a pattern at the end of the day.

sqeaky 9y ago

You are right about the majority of what you said, but I am pedantically picking on one point. CPU cores are getting faster, but they aren't doing it with clock speed, they are dispatching more instructions per cycle or otherwise making the work faster.

kllrnohj 9y ago
IPC gains per generation are vanishingly tiny, if they exist at all. Skylake -> Kaby Lake, for example, had no IPC improvements at all. A very small clock bump to the various tiers was it. Even if you look over a large generation gap there's only a ~20% IPC improvement going from an i7-2600K to an i7-7700K ( https://www.hardocp.com/article/2017/01/13/kaby_lake_7700k_vs_sandy_bridge_2600k_ipc_review/ https://www.hardocp.com/article/2017/01/13/kaby_lake_7700k_v... ) 6 years & a shrink from 32nm to 14nm and all it can muster is +20%. Cores are just not getting faster by any meaningful amount.
- happycube 9y ago
  Yeah - I remember when five year old CPUs were basically useless! (Kaby Lake is basically a new stepping of Skylake - if intel wasn't having problems with new process nodes it likely wouldn't have been released at all, and if it was it would've been used for a one-off chip in the same generation ala the 4770K)
- sqeaky 9y ago
  My compiles have gotten more than 20% faster so something is making my newere machines faster than my older machines. That it is not 10x as fast I blame on AMD for not being as competitive as they could have been.

jononor 9y ago

The efficiency hit is very dependent on how large your computation chunks are. If the computation per message batch is on order of 100 ms, it would be <10% loss.

kllrnohj 9y ago
Assuming very small messages that are rarely sent then yes, the hit of multi process is not going to be your biggest issue.

breatheoften 9y ago

Moving data between threads is only free to the extent that synchronization is free. Maybe you could say that moving immutable data between threads is free but I don't think you can say its free in general ... Doing so significantly undersells the complexity that comes with shared memory concurrency.

kllrnohj 9y ago
You seem to be conflating moving with sharing. Moving between threads is always free[1] regardless of if it's mutable or immutable, and there's no concurrency issues at all since it's a move. Move means the sender no longer has a reference. As in, std::move, rust's ownership transfer, webworker's transferables, etc... 1: Yes there's a single synchronize point where the handoff happens, but this is part of sending a message at all. It's also independent to the size & complexity of the payload itself when we're talking multi-threaded instead of multi-process. You have that exact same sync point that costs the exact same regardless of whether your message consists of a single byte or a multi-gigabyte structure.
- breatheoften 9y ago
  Ah I see -- Can you actually describe ownership in python sufficiently well to be able to describe this move operation for any useful python data structures?
  - kllrnohj 9y ago
    Generically ownership is simply who has a reference to the object. So if a.foo = b, then a 'owns' b. A 'move' is simply handing a different object the reference, then dropping your own reference. For example: a.foo = b // a 'owns' b c.foo = a.foo // a & c share b, however if the next line is: a.foo = None // a has 'moved' b to c, since c now has the only reference to b. Some languages have codified this to make the contract part of the language, but it doesn't need any first-class language support. It's just a pattern at the end of the day.