3 ms·
"Python async sucks", then rants about django "Python doesn't have native async file I/O." - like almost everybody, as "sane" file async IO on Linux is somehow
by JackSlateur 1y ago
"Python async sucks", then rants about django
"Python doesn't have native async file I/O." - like almost everybody, as "sane" file async IO on Linux is somehow new (io_uring)
Anyway ..
- jitl 1y agolibuv has provided an async interface for io using a worker thread pool for a decade, no dependency on io_uring required. I guess the threadpool they mention that aiofiles uses is written in python, so it gives concurrency but retains the GIL, so no parallelism. Node's libuv async stuff moves all the work off the main thread into c/c++ land until results are ready, only when dealing with the completed data read event does it re-enter the NodeJS "GIL" JavaScript thread.
- hinkley 1y agoLibuv has had io_uring integration for almost 18 months if you’re not on an old kernel or old hardware. They claim about an 8x improvement in speed.
- jitl 1y agothat's great! just saying io_uring has never been a required dependency to write application logic that avoids blockage on reads.
- danudey 1y agoTo be clear: libuv has had the ability to offload (some?) I/O operators to io_uring since v1.45.0, from 2023, and that's the 8x speed improvement. 2024 is when node.js seemed to enable (or rather, stop disabling) io_uring by default in its own usage of libuv.
- hinkley 1y agoYeah if you look at the libuv release history there’s been a lot of adding and subtracting since then. It’s clearly not all settled but there are chunks.
- morshu9001 1y agoPython async does suck though
- kurtis_reed 1y agoWhy?
- morshu9001 1y agoHistorically the main way was Py threads that have all the downsides of OS threads (overhead + data races) without the parallelism advantage (due to GIL), with a confusing API on top. There was no cooperative multitasking until asyncio, at which point it was too late. There are competing solutions now, and many users are thoroughly confused. And I'm guessing the reason they didn't do greenthreading is it'd severely complicate working with C/native libs.
- antirez 1y agoSane async does not need io_uring. All the non-linear-time select(2) alternatives are good enough.
- JackSlateur 1y agoHow do you do that ? More details would be welcome