3 ms·

First of all, see this pic: https://pbs.twimg.com/media/COLLg0TUAAA4j79.jpg:large https://pbs.twimg.com/media/COLLg0TUAAA4j79.jpg:large asyncio doesn't provide

by 1st1 11y ago

First of all, see this pic: https://pbs.twimg.com/media/COLLg0TUAAA4j79.jpg:large https://pbs.twimg.com/media/COLLg0TUAAA4j79.jpg:large asyncio doesn't provide any nio abstractions for files because (a) it's not really needed, and (b) there is no easy way to implement it. (a) basically you shouldn't expect your code to block on disk io. But even if it does block for a very short amount of time it's probably fine. (b) one way to implement nio for files is to use a threadpool. Maybe we'll add this in later versions of asyncio, but it will require to write some pretty low level code in C (and reimplement big chunks of asyncio in C too). Another option is to use modern APIs like aio in Linux, but as far as I know almost nobody uses it for real. Bottom line -- you don't need coroutines or asyncio to do file io. What you need asyncio (and frameworks like aiohttp) is to do network programming in Python efficiently.

sonicrocketman 11y ago
After playing with Asyncio more, I certainly see your point. I guess my larger point was that, even with the new async/await syntax, Python's general multitasking is still rough and incomplete imo. I thought async/await was a solution, but it's not. I just imagine it being so much simpler.
athrun 11y ago
"you shouldn't expect your code to block on disk io" uh?! In what world are you living in? Blocking the main thread of your app on disk I/O makes for a poor user experience. Disk I/O is a big deal for a whole range of applications. I would even say almost all apps have to deal with disk I/O at some point, as opposed to network I/O.
beagle3 11y ago
> basically you shouldn't expect your code to block on disk io. But even if it does block for a very short amount of time it's probably fine. No, it's very often not fine. Magnetic disks, still the norm for many, and definitely with large storage, often go as low as 5KB/s for random access (or even sequential access to very fragmented files). Reading a 1MB file can easily take 5-10 seconds in some setups - which is not acceptable for any interactive service. It's not fine for a web server to not service any requests for 5 seconds. > Another option is to use modern APIs like aio in Linux, but as far as I know almost nobody uses it for real. Anyone I know who tried came back screaming. There is no way to do an async file open, for example - which means that if you rely on aio, you can block for 10 minutes waiting for an NFS or SMB mounted file to open. The only sane, portable thing to do for Unix/Posix is use a threadpool for async file io - or just use something like libuv which already abstracted async operations this way.

by 1st1 11y ago

sonicrocketman 11y ago

After playing with Asyncio more, I certainly see your point. I guess my larger point was that, even with the new async/await syntax, Python's general multitasking is still rough and incomplete imo. I thought async/await was a solution, but it's not. I just imagine it being so much simpler.

athrun 11y ago

"you shouldn't expect your code to block on disk io" uh?! In what world are you living in? Blocking the main thread of your app on disk I/O makes for a poor user experience. Disk I/O is a big deal for a whole range of applications. I would even say almost all apps have to deal with disk I/O at some point, as opposed to network I/O.

beagle3 11y ago

> basically you shouldn't expect your code to block on disk io. But even if it does block for a very short amount of time it's probably fine. No, it's very often not fine. Magnetic disks, still the norm for many, and definitely with large storage, often go as low as 5KB/s for random access (or even sequential access to very fragmented files). Reading a 1MB file can easily take 5-10 seconds in some setups - which is not acceptable for any interactive service. It's not fine for a web server to not service any requests for 5 seconds. > Another option is to use modern APIs like aio in Linux, but as far as I know almost nobody uses it for real. Anyone I know who tried came back screaming. There is no way to do an async file open, for example - which means that if you rely on aio, you can block for 10 minutes waiting for an NFS or SMB mounted file to open. The only sane, portable thing to do for Unix/Posix is use a threadpool for async file io - or just use something like libuv which already abstracted async operations this way.