5 ms·

Fully agree. People put far too much emphasis & expectations on the "free" part in "free multithreading".

slt2021 3y ago
even if they get "free multithreading" with no-GIL, their system eventually will overgrow one beefy machine and will need to be deployed across a fleet of 10/100/1000 machines. at which point you lose benefit of no-GIL, because you now have to introduce redis and kafka into the system
- dragonwriter 3y ago
  > even if they get “free multithreading” with no-GIL, their system eventually will overgrow one beefy machine Why? Yeah, if you are building a system that is, say, serving web requests, and have an internet scale potential market, success might mean that. Not every system works that way. A simulation system with defined parameters doesn’t grow in scale if it becomes more popular, you just have more people running isolated instances that don’t depend on each other. Plenty of other applications scale that way rather than the “SaaS that serves ever more clients” way.
- miraculixx 3y ago
  I kept making this point as well as the other arguments above (and others did too) in the Core Dev discussion group. Unfortuately to no avail. To be sure I am not a core dev.
  - matsemann 3y ago
    Well I don't agree that just because one needs >1 servers, no-gil is suddenly useless. Still lots of complexity and awkwardness that can be avoided if you can do threading instead of processes. Like Promotheus scraping from a non-webserver python app is a pita, as you need a new process and lots of communication, vs just plug and play as in other languages. Or just the insane resource usage. Had a java app serving multiple orders of magnitude more customers running on a few containers. Our current python app needs multiple deployments with different entry points, and about 15x the amount of containers.
    - slt2021 3y ago
      It is not fair to compare CPython (which is on purpose not optimized, only a reference implementation of interpretable scripting language without any focus on performance) to OpenJDK - an arguably state of the art compiled bytecode VM with JIT and AOT compilers available, with decades and many $millions poured into runtime/JIT/GC/etc research and optimization
      - lozenge 3y ago
        "on purpose not optimized, only a reference implementation of interpretable scripting language without any focus on performance" That policy is over. As the last years have shown, no alternative implementation can get off the ground due to C extensions and compatibility concerns, and CPython is now relied on for many large applications. It no longer makes sense to prioritise a simple implementation over performance.
      - matsemann 3y ago
        Why is it not fair? I need to choose a tool, how it came to be is irrelevant for me.
  - fred123 3y ago
    Do you think Meta (Instagram) are pushing GIL removal and Cinder for no reason? They clearly have that scale and still benefit from faster single machine performance
    - miraculixx 3y ago
      Yes they will benefit from this. Unfortunately most everyone else will suffer.
- mlyle 3y ago
  I think this argument presumes that everything is the sort of problem that maps well to redis and kafka. Scientific computing doesn't. And while things like numpy might lower contention on the GIL a bit, it's not a cure-all. Finely-grained locks are useful. Even when you end up scaling between machines, it can be useful to have many threads in one memory space to maximize what you get out of one machine. We're moving up to hundreds of cores; Python often being stuck only being able to use a couple while tightly coupling state has been unfortunate.
- fred123 3y ago
  Why? On AWS you can rent a 24 TB, 500 core machine. Almost all problems are smaller than that so don’t need to scale to more than one machine. Building applications that run on multiple machines is at least one order of magnitude more complex and thus slower (in development velocity), so needlessly building an application to work distributedly is just bad engineering.
  - tgv 3y ago
    Dear Lord, is that real? It surely would not be appropriate to compensate for bad performance in a Python app.
    - imtringued 3y ago
      I wonder if SAP Hana is written in python. Because EC2 Nitro is certified for SAP.
- Dylan16807 3y ago
  Most systems don't grow forever, and can stay on one machine. And "one beefy machine" has a very high limit, so by the time you actually outgrow it you usually have tons of resources available to help rewrite things.
- xmcqdpt2 3y ago
  Yes well if you are distributing to N machines, you probably want to use all M cores on each of those machines. You'll still get a performance advantage from multi-threading. You might think that you can simply spin M processes per machine instead but now you have N*M servers instead of N servers that are M times faster. In many cases this means you have significantly higher overheads: slower startups, a lot more RAM usage, more network IO etc. Outside of a few embarrassingly parallel problem, two-level parallelism is usually the highest performance approach.

slt2021 3y ago

even if they get "free multithreading" with no-GIL, their system eventually will overgrow one beefy machine and will need to be deployed across a fleet of 10/100/1000 machines. at which point you lose benefit of no-GIL, because you now have to introduce redis and kafka into the system

dragonwriter 3y ago
> even if they get “free multithreading” with no-GIL, their system eventually will overgrow one beefy machine Why? Yeah, if you are building a system that is, say, serving web requests, and have an internet scale potential market, success might mean that. Not every system works that way. A simulation system with defined parameters doesn’t grow in scale if it becomes more popular, you just have more people running isolated instances that don’t depend on each other. Plenty of other applications scale that way rather than the “SaaS that serves ever more clients” way.
miraculixx 3y ago
I kept making this point as well as the other arguments above (and others did too) in the Core Dev discussion group. Unfortuately to no avail. To be sure I am not a core dev.
- matsemann 3y ago
  Well I don't agree that just because one needs >1 servers, no-gil is suddenly useless. Still lots of complexity and awkwardness that can be avoided if you can do threading instead of processes. Like Promotheus scraping from a non-webserver python app is a pita, as you need a new process and lots of communication, vs just plug and play as in other languages. Or just the insane resource usage. Had a java app serving multiple orders of magnitude more customers running on a few containers. Our current python app needs multiple deployments with different entry points, and about 15x the amount of containers.
  - slt2021 3y ago
    It is not fair to compare CPython (which is on purpose not optimized, only a reference implementation of interpretable scripting language without any focus on performance) to OpenJDK - an arguably state of the art compiled bytecode VM with JIT and AOT compilers available, with decades and many $millions poured into runtime/JIT/GC/etc research and optimization
    - lozenge 3y ago
      "on purpose not optimized, only a reference implementation of interpretable scripting language without any focus on performance" That policy is over. As the last years have shown, no alternative implementation can get off the ground due to C extensions and compatibility concerns, and CPython is now relied on for many large applications. It no longer makes sense to prioritise a simple implementation over performance.
    - matsemann 3y ago
      Why is it not fair? I need to choose a tool, how it came to be is irrelevant for me.
- fred123 3y ago
  Do you think Meta (Instagram) are pushing GIL removal and Cinder for no reason? They clearly have that scale and still benefit from faster single machine performance
  - miraculixx 3y ago
    Yes they will benefit from this. Unfortunately most everyone else will suffer.
mlyle 3y ago
I think this argument presumes that everything is the sort of problem that maps well to redis and kafka. Scientific computing doesn't. And while things like numpy might lower contention on the GIL a bit, it's not a cure-all. Finely-grained locks are useful. Even when you end up scaling between machines, it can be useful to have many threads in one memory space to maximize what you get out of one machine. We're moving up to hundreds of cores; Python often being stuck only being able to use a couple while tightly coupling state has been unfortunate.
fred123 3y ago
Why? On AWS you can rent a 24 TB, 500 core machine. Almost all problems are smaller than that so don’t need to scale to more than one machine. Building applications that run on multiple machines is at least one order of magnitude more complex and thus slower (in development velocity), so needlessly building an application to work distributedly is just bad engineering.
- tgv 3y ago
  Dear Lord, is that real? It surely would not be appropriate to compensate for bad performance in a Python app.
  - imtringued 3y ago
    I wonder if SAP Hana is written in python. Because EC2 Nitro is certified for SAP.
Dylan16807 3y ago
Most systems don't grow forever, and can stay on one machine. And "one beefy machine" has a very high limit, so by the time you actually outgrow it you usually have tons of resources available to help rewrite things.
xmcqdpt2 3y ago
Yes well if you are distributing to N machines, you probably want to use all M cores on each of those machines. You'll still get a performance advantage from multi-threading. You might think that you can simply spin M processes per machine instead but now you have N*M servers instead of N servers that are M times faster. In many cases this means you have significantly higher overheads: slower startups, a lot more RAM usage, more network IO etc. Outside of a few embarrassingly parallel problem, two-level parallelism is usually the highest performance approach.