4 ms·
using threads for file IO is a hammer approach imho. Often times the price you pay to send the job to the queue, yield for another coroutine, wait for IO task
by halayli 6y ago
using threads for file IO is a hammer approach imho.
Often times the price you pay to send the job to the queue, yield for another coroutine, wait for IO task to finish, receive a notification back from the thread pool and resume is higher than doing the blocking call. Under high load it might have some small benefits but my hunch is that it's unlikely to buy you much.
Luckily, io_uring officially made it to the linux kernel few months ago and can solve these problems in a much more efficient manner. *BSD solved this problem long time ago, in 2002, when kqueue was introduced.
- deleted 6y ago[deleted]
- ridiculous_fish 6y agoNote this is specifically about opening and closing files, which may be on network mounts. kqueue has no support for this, io_uring just very recently got it. So there's no alternative to threads today.
- halayli 6y agokqueue does support this via EVFILT_VNODE. io_uring is in linux as of kernel 5.1(march 2020). alternatives are available but might not be viable for everyone.
- ridiculous_fish 6y agoMy understanding is that kqueue can notify for file events like "readable" or "opened," but cannot actually initiate reading or opening.
- halayli 6y agokqueue is not responsible for initiation. It's responsible to notify you about events. when you open(2) a file with O_NONBLOCK flag, you can register it with kqueue to let you know when it's readable/writable.