4 ms·

BearSSL is a great library, I also implemented TLS support with it recently in my low-resource rust webserver project wwwee [1]. It's one of the nicest designed

BearSSL is a great library, I also implemented TLS support with it recently in my low-resource rust webserver project wwwee [1]. It's one of the nicest designed C libraries I've ever seen. I hope more projects start using it so it gets out of beta. For static files, I ended up using the aio syscalls in Linux as they work quite well for reading files on a supporting filesystem (mainly ext4, xfs). I also couldn't figure out how to guarantee that sendfile doesn't block, but didn't look into it too deeply. Very nice project, seems very mature. 1: https://github.com/bwindels/wwwee/ https://github.com/bwindels/wwwee/

acidx 8y ago
Thanks! I don't think there's a way to guarantee that sendfile() won't block on Linux. On FreeBSD, you can specify a "go as much as you can without blocking, and then return" flag; Linux doesn't have a flags parameter for sendfile() (maybe it needs a sendfile2() syscall with that?). You can maybe use readahead() and keep the file open, but that may also block as it does its work behind the scenes. A thread just for this kind of blocking calls would be nice. But even then, this doesn't seem sufficient; so you might want to use multiplex sendfile() calls in bursts. Also, yeah, Linux aio syscalls are interesting, but they're quite tricky to use and not very portable. I'll check out your project, thanks for linking to it -- should be useful reading material while I learn Rust.
- bwindels 8y ago
  Right, wrt sendfile, I remember coming to that conclusion as well. I really wanted a thread-less design, mostly for simplicity's sake and to support single core SOCs well. Is that what you do in Lwan for large files? Use a dedicated thread for socket writes with sendfile? I guess mmap + madvise could also help, but there is just no way to know for sure, so you sort of have to assume the worst? Which ever way you do it, it's so easy to end up blocking without realizing, strace is great here to see how much time you actually spend in the kernel. How much the aio calls block is highly filesystem-dependent. I was hoping to use btrfs for my project, but when testing aio support, I saw my demo was spending around 80% in io_submit and less than 10% in epoll_wait, where as with ext4 it was more than the opposite. Interesting stuff, and doing it in Rust was a pleasant experience I must say, although if you are trying hard to avoid copying and allocating, the borrow checker can get a bit naggy. Worth it though IMO, and should improve in the future.
  - acidx 8y ago
    No -- in Lwan, files that will be sent with sendfile() are kept open for a while; when they're first open, the same thread that opened (and this might block) will call readahead(). Files are sent in 512kB bursts. It might use mmap()+madvise(), but only for small files (<16KB)... but, yeah, there's not a lot of control here. I'm not that worried, though; this is but a toy. :) Registered I/O (RIO) or I/O Completion Ports (IOCP) in Windows seems to be better in this regard, but I don't have a lot of experience in that platform. Trent Nelson, from the PyParallel project, has a few talks about it; it's quite neat.

acidx 8y ago

Thanks! I don't think there's a way to guarantee that sendfile() won't block on Linux. On FreeBSD, you can specify a "go as much as you can without blocking, and then return" flag; Linux doesn't have a flags parameter for sendfile() (maybe it needs a sendfile2() syscall with that?). You can maybe use readahead() and keep the file open, but that may also block as it does its work behind the scenes. A thread just for this kind of blocking calls would be nice. But even then, this doesn't seem sufficient; so you might want to use multiplex sendfile() calls in bursts. Also, yeah, Linux aio syscalls are interesting, but they're quite tricky to use and not very portable. I'll check out your project, thanks for linking to it -- should be useful reading material while I learn Rust.

bwindels 8y ago
Right, wrt sendfile, I remember coming to that conclusion as well. I really wanted a thread-less design, mostly for simplicity's sake and to support single core SOCs well. Is that what you do in Lwan for large files? Use a dedicated thread for socket writes with sendfile? I guess mmap + madvise could also help, but there is just no way to know for sure, so you sort of have to assume the worst? Which ever way you do it, it's so easy to end up blocking without realizing, strace is great here to see how much time you actually spend in the kernel. How much the aio calls block is highly filesystem-dependent. I was hoping to use btrfs for my project, but when testing aio support, I saw my demo was spending around 80% in io_submit and less than 10% in epoll_wait, where as with ext4 it was more than the opposite. Interesting stuff, and doing it in Rust was a pleasant experience I must say, although if you are trying hard to avoid copying and allocating, the borrow checker can get a bit naggy. Worth it though IMO, and should improve in the future.
- acidx 8y ago
  No -- in Lwan, files that will be sent with sendfile() are kept open for a while; when they're first open, the same thread that opened (and this might block) will call readahead(). Files are sent in 512kB bursts. It might use mmap()+madvise(), but only for small files (<16KB)... but, yeah, there's not a lot of control here. I'm not that worried, though; this is but a toy. :) Registered I/O (RIO) or I/O Completion Ports (IOCP) in Windows seems to be better in this regard, but I don't have a lot of experience in that platform. Trent Nelson, from the PyParallel project, has a few talks about it; it's quite neat.