3 ms·

Yes, I mean the async keyword fixes the hijacking although the hijacked version is still there for compatibility. So much for Python 3 breaking stuff for the p

Yes, I mean the async keyword fixes the hijacking although the hijacked version is still there for compatibility. So much for Python 3 breaking stuff for the purpose of fixing old cruft. The tragedy is the whole Python 3 debacle, and if they were going to make a break like that, they should have introduced BEAM-like concurrency instead of async. For reasons I don't understand, the Python community seems to have an intense phobia and hatred towards threads. I've always used threads instead of async though, and by writing in an Erlang-like style (message passing and no shared mutable data) you can keep things reliable. This has worked ok for me with 1000s of threads, though not millions or anything like that. Python by now is mostly legacy cruft, it feels to me sometimes. Everything else of course has its own problems.

bormaj 2y ago
As someone who started learning to code with C++ followed by Python and then Rust, proper threading approaches weren't drilled into my head until I started working with Rust. I agree with you that threading with message passing is a safer approach. Threading does add a level of complexity that isn't always necessary. AFAIK it's not outlined in a PEP, but my intuition is that Python was designed to be accessible and "simple". It wasn't built to be a systems programming language. So for all its faults with the GIL I don't think it should be a major point of contention. For any task it's always about choosing the right tool for the job. There is considerable effort at the moment to remove the GIL in future versions and implement better threading support. At 3.11/3.12, it has come a long way in terms of performance.
hot_gril 2y ago
The problem with threads in Python is they're 1:1 with OS threads in every major implementation. It's not like Golang where you have greenthreading, and you don't even get all the benefits of regular threads due to the GIL. It's been too long since I've used Erlang, but the way I remember it, the concurrency is almost like using RPCs. Which is great in some applications, but Python users want shared mutable data for simple things and will make microservices for larger systems. For a simple language, Python has so many different ways to do concurrency, some of which are misleading to new users. Event loop ("async") would've been nice and simple from the start, but they were late to that party. Everything vaguely related to packaging is also confusing, from the __init__.py files to the PIP/Conda/whatever stuff. They had a whole breaking Py2->3 transition and still didn't fix that.
- throwaway81523 2y ago
  The GIL is an implementation artifact mostly due to the reference counting memory management. Besides phobia towards threads, the Python crowd also seems phobic towards tracing garbage collection. There is an ongoing project to make CPython run without the GIL yet still use reference counting. I wonder how many bugs this will surface or introduce. Most concurrent Python code that I've written has not been cpu intensive, FWIW. Just an application talking to a bunch of i/o ports or sockets, but mostly idle. My CPU intensive stuff has tended to be single threaded so I can just launch instances in separate processes. YMMV. Yeah the tooling situation is crazy enough that I don't begin to understand it.
  - hot_gril 2y ago
    Yeah, Python is made for non-CPU-intensive, IO-bound concurrent code. But that use case is a great fit for async/event-loop.

bormaj 2y ago

As someone who started learning to code with C++ followed by Python and then Rust, proper threading approaches weren't drilled into my head until I started working with Rust. I agree with you that threading with message passing is a safer approach. Threading does add a level of complexity that isn't always necessary. AFAIK it's not outlined in a PEP, but my intuition is that Python was designed to be accessible and "simple". It wasn't built to be a systems programming language. So for all its faults with the GIL I don't think it should be a major point of contention. For any task it's always about choosing the right tool for the job. There is considerable effort at the moment to remove the GIL in future versions and implement better threading support. At 3.11/3.12, it has come a long way in terms of performance.

hot_gril 2y ago

The problem with threads in Python is they're 1:1 with OS threads in every major implementation. It's not like Golang where you have greenthreading, and you don't even get all the benefits of regular threads due to the GIL. It's been too long since I've used Erlang, but the way I remember it, the concurrency is almost like using RPCs. Which is great in some applications, but Python users want shared mutable data for simple things and will make microservices for larger systems. For a simple language, Python has so many different ways to do concurrency, some of which are misleading to new users. Event loop ("async") would've been nice and simple from the start, but they were late to that party. Everything vaguely related to packaging is also confusing, from the __init__.py files to the PIP/Conda/whatever stuff. They had a whole breaking Py2->3 transition and still didn't fix that.

throwaway81523 2y ago
The GIL is an implementation artifact mostly due to the reference counting memory management. Besides phobia towards threads, the Python crowd also seems phobic towards tracing garbage collection. There is an ongoing project to make CPython run without the GIL yet still use reference counting. I wonder how many bugs this will surface or introduce. Most concurrent Python code that I've written has not been cpu intensive, FWIW. Just an application talking to a bunch of i/o ports or sockets, but mostly idle. My CPU intensive stuff has tended to be single threaded so I can just launch instances in separate processes. YMMV. Yeah the tooling situation is crazy enough that I don't begin to understand it.
- hot_gril 2y ago
  Yeah, Python is made for non-CPU-intensive, IO-bound concurrent code. But that use case is a great fit for async/event-loop.