4 ms·
Not a question, but I made a similar tool in c++[1], for Linux only using io_uring, and a blog post explaining its internals [2]. I'll definitely have a look so
by wheybags 5y ago
Not a question, but I made a similar tool in c++[1], for Linux only using io_uring, and a blog post explaining its internals [2]. I'll definitely have a look some time soon, I'd be interested to see how performance compares (I gathered from some other comments here that you're using blocking io in threads?)
1: https://github.com/wheybags/wcp https://github.com/wheybags/wcp
2: https://wheybags.com/blog/wcp.html https://wheybags.com/blog/wcp.html
- Svetlitski 5y agoSounds super interesting, I'll definitely check out the blog post. Yes, fcp is indeed using blocking io in threads. In an earlier prototype I was using async (i.e. M on N threading), but was running into issues with exhausting file descriptors. Because of this (and because writing async Rust is quite difficult) I settled on using blocking io in threads. Perhaps in a future version I'll try to implement the io_uring approach for Linux.
- wheybags 5y agoAh, I also tried unlocking the file descriptor 1024 limit, which worked fine most of the time, but didn't actually give me a perf boost in any of my tests. I got rid of it in the end because it caused problems when copying from an smb mount. iirc I exhausted the file descriptor limit on the samba server.