4 ms·

You can recompile libc if you want to change the limit, or at least you could in the past.

o11c 1y ago
You don't have to recompile, just do the following (at least on glibc): #include <sys/types.h> // pull in initial definition of __FD_SETSIZE #undef __FD_SETSIZE #define __FD_SETSIZE 32768 // or whatever #include <sys/select.h> // won't include the internal <bits/types.h> again This is a rare case when `-Wsystem-headers` is useful to enable (and these days system headers are usually pretty clean) - it will catch if you accidentally define `__FD_SETSIZE` before the system does. Note that `select` is still the nicest API in a lot of ways - `poll` wastes space gratuitously, `epoll` requires lots of finicky `modify` syscalls, and `io_uring` is frankly not sane. That said: * if you're only dealing with a couple FDs, use `poll`. * it's not that hard to take a day and think about epoll write buffer management. You need to consider every combination of: epoll state is/isn't checking writability (you want to only change this lazily) on the previous/current iteration, was there nothing/something in the write buffer? prior actual write was would-block/actually-incomplete/spuriously-incomplete/complete current actual write ends up would-block/actually-incomplete/spuriously-incomplete/complete There are many "correct" answers, but I suspect the optimal answer for epoll is something like: initially, write optimistically (and do this before the wait). If you fail to write anything at all, enable the kernel flag. For FDs that you've previously enabled the flag for, if you don't have anything to write this time, disable the flag; otherwise, don't actually write until after the wait (it is guaranteed to return immediately if the write would be allowed, after all, but you'll also get other events that happen to be ready). If you trust your event handlers to return quickly, you can defer any indicated writes until the next wait, otherwise do them before handling events. You can see why people still use `select`.
- o11c 1y ago
  Checking other libcs (note that "edit the header" is not that difficult to automate): bionic - must edit the header dietlibc - must edit the header glibc - undocumented but reliable, see the dance in the original post klibc - must edit <linux/posix_types.h> (which, note, sabotages glibc) MUSL - must edit the header newlib - documented in header, just `#define FD_SETSIZE` before you `#include <sys/select.h>` uclibc - as glibc (since it's a distant fork). Note that `poll.c` for old uclinux kernels is implemented in terms of `select` with dynamic `fd_set` sizing logic! freebsd - properly documented, just `#define FD_SETSIZE` first netbsd - properly documented, just `#define FD_SETSIZE` first openbsd - documented just in the header now (formerly in the man page too), just `#define FD_SETSIZE` first solaris - properly documented, just `#define FD_SETSIZE` first macos - properly documented, just `#define FD_SETSIZE` first winsock - properly documented, just `#define FD_SETSIZE` first, but note the API is not actually the same

o11c 1y ago

You don't have to recompile, just do the following (at least on glibc): #include <sys/types.h> // pull in initial definition of __FD_SETSIZE #undef __FD_SETSIZE #define __FD_SETSIZE 32768 // or whatever #include <sys/select.h> // won't include the internal <bits/types.h> again This is a rare case when `-Wsystem-headers` is useful to enable (and these days system headers are usually pretty clean) - it will catch if you accidentally define `__FD_SETSIZE` before the system does. Note that `select` is still the nicest API in a lot of ways - `poll` wastes space gratuitously, `epoll` requires lots of finicky `modify` syscalls, and `io_uring` is frankly not sane. That said: * if you're only dealing with a couple FDs, use `poll`. * it's not that hard to take a day and think about epoll write buffer management. You need to consider every combination of: epoll state is/isn't checking writability (you want to only change this lazily) on the previous/current iteration, was there nothing/something in the write buffer? prior actual write was would-block/actually-incomplete/spuriously-incomplete/complete current actual write ends up would-block/actually-incomplete/spuriously-incomplete/complete There are many "correct" answers, but I suspect the optimal answer for epoll is something like: initially, write optimistically (and do this before the wait). If you fail to write anything at all, enable the kernel flag. For FDs that you've previously enabled the flag for, if you don't have anything to write this time, disable the flag; otherwise, don't actually write until after the wait (it is guaranteed to return immediately if the write would be allowed, after all, but you'll also get other events that happen to be ready). If you trust your event handlers to return quickly, you can defer any indicated writes until the next wait, otherwise do them before handling events. You can see why people still use `select`.

o11c 1y ago
Checking other libcs (note that "edit the header" is not that difficult to automate): bionic - must edit the header dietlibc - must edit the header glibc - undocumented but reliable, see the dance in the original post klibc - must edit <linux/posix_types.h> (which, note, sabotages glibc) MUSL - must edit the header newlib - documented in header, just `#define FD_SETSIZE` before you `#include <sys/select.h>` uclibc - as glibc (since it's a distant fork). Note that `poll.c` for old uclinux kernels is implemented in terms of `select` with dynamic `fd_set` sizing logic! freebsd - properly documented, just `#define FD_SETSIZE` first netbsd - properly documented, just `#define FD_SETSIZE` first openbsd - documented just in the header now (formerly in the man page too), just `#define FD_SETSIZE` first solaris - properly documented, just `#define FD_SETSIZE` first macos - properly documented, just `#define FD_SETSIZE` first winsock - properly documented, just `#define FD_SETSIZE` first, but note the API is not actually the same