3 ms·
If anyone is wondering why the threaded implementation was slower > pool = ThreadPool(40)
by No1 5y ago
If anyone is wondering why the threaded implementation was slower
> pool = ThreadPool(40)
- akvadrako 5y agoMany people still don't understand you can run millions of threads on Linux on a beefy server. The bottleneck with python will always be the CPU doing something with those requests.
- remram 5y agoTo be fair they did try multiple concurrency factors for asyncio and found that 60 was fastest, so this is not too far. Hopefully.
- AtlasBarfed 5y agoHow is python doing a thread pool with the GIL?
- anamax 5y ago> How is python doing a thread pool with the GIL? All but one of the runnable threads are waiting on the GIL. The "not runnable" threads are waiting on some lock or a blocking system call.
- AtlasBarfed 5y agoSo it's a single thread event loop? which means 1 cpu utilization? I get a lot of people have gotten mileage out of event loop scaling over the last 10-15 years (that's the trick of nginx, node.js, etc, right?), but even raspberry pis are quad core?
- anamax 5y agoSome C extensions run even when they don't have the lock. Also, the OS gets to run. If you have a lot of parallel computation, Python may not be a great choice. However, if you're managing a lot of IO with not much computation, Python is perfectly reasonable.