7 ms·

No new code should be using select(). Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. I've personally had to de

No new code should be using select(). Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. I've personally had to debug and fix crashes due to this assumption at multiple jobs. Back at Hostway decades ago we had Apache+SSL instances randomly crashing which turned out to be due to something opening /dev/random and using select() on the returned file descriptor in a .so. This only happened when the machine had enough virtual hosts to exceed FD_SETSIZE on the spurious open of /dev/random. At Bizanga we had random crashes in their "carrier grade mail server" software, which would link in proprietary modules from vendors like Symantec for realtime virus scanning before even accepting messages for delivery. This stuff ran in-process for performance reasons, just in separate threads. Being "carrier grade" meant thousands of concurrent connections -> thousands of open files. Those vendor-provided plugins would phone home behind the scenes for virus db updates. Turned out they were using select() on the socket connected for the update, if there were enough active sessions at the time, the socket's value would exceed FD_SETSIZE and boom: segfault. People tend to forget that it's the value of the fds you pass to select() which can't exceed FD_SETSIZE, not just the quantity of file descriptors. It could be just one file descriptor, but if it's a high enough value exceeding FD_SETSIZE, it will segfault. The value is used as an index into a fixed size array. I ended up making an execution wrapper @ Bizanga to launch everything with >FD_SETSIZE file descriptors already opened once I saw the vendor modules were linking select(). We found numerous crashes this way, without waiting for them to occur in production.

ridiculous_fish 5y ago
Is there a portable alternative to pselect()? There's ppoll(), but it's missing on some platforms.
- wahern 5y ago
  At this point I believe ppoll has been added to every mainstream Unix except macOS. It's also been accepted for the next POSIX revision. For many years pselect on macOS didn't actually fix the race condition, notwithstanding its UNIX03 certification. pselect was just a naive wrapper around select that didn't atomically unmask signals. So I ended up having to create a kqueue-based implementation for use on macOS and stragglers like OpenBSD which at that time lacked pselect entirely. See https://github.com/wahern/cqueues/blob/87f14f9/src/cqueues.c#L2489 https://github.com/wahern/cqueues/blob/87f14f9/src/cqueues.c... I can't remember if pselect has been fixed on macOS. The same approach could be used to emulate ppoll on macOS. Though epoll, kqueue, and Solaris ports are so similar that you can easily wrap all of them behind a thin shim. See https://github.com/wahern/cqueues/blob/87f14f9/src/lib/kpoll.c https://github.com/wahern/cqueues/blob/87f14f9/src/lib/kpoll... (Caution: it's an incomplete refactor of a similar shim I used in some non-open source code. The original code is even simpler and has seen extensive use. The refactored shim should work but is untested.) AIX is really the only mainstream Unix lacking an API w/ equivalent semantics. AIX has a better-than-poll interface, but it doesn't work anything like the others. Notably, epoll, kqueue, and Solaris ports control descriptors are all pollable in their own right, permitting you to nest different [readiness polling-based] event loop frameworks.
- jlokier 5y ago
  There is a way to simulate it portably, but it's a little complicated. You can use the self-pipe trick portably on POSIX systems I believe. (There is also the sigsetjmp/siglongjmp trick on most platforms, but I remember it breaking on some version of Cygwin.) The self-pipe trick is where you create a pipe(), add the read side to select(), poll() or other syscall, and have your signal handlers write a byte to the write side. When reading the pipe to flush this state, remember to read until it's empty because there may have been multiple signals. Reading 2 bytes and looping if you get 2 will get this done usually in one syscall. It's best in principle to make the pipe non-blocking so the writes can never block if the pipe fills. This is basically the portable equivalent to Linux's eventfd. (If you're worried about out-of-memory preventing non-blocking write of a single byte, you can close the pipe instead, but this causes several complications because it's not safe to close the same fd twice, and you may have signals interrupt other signal handlers, so race hazards around closing the fd twice must be protected against by blocking signals inside the signal handler and using more flags.) You can optimise self-pipe by setting a write-enable flag just before select() etc and clearing it afterwards, with the signal handlers only writing a byte (or closing the pipe) when the flag is set, and setting a separate flag to say that the signal occurred, which is checked between setting the write-enable flag and before the select() etc syscall. Check the signal-happened flag after select() etc and if it's set, read the pipe to flush even if it doesn't show as readable in the fd set, because the signal could have happened later. A further optimisation is for the signal handler to clear write-enable when writing the pipe, so subsequent signals don't do it again. Don't assume this means there can only be one byte in the pipe, because nested signals introduce a race hazard to writing and clearing write-enable atomically. The sigsetjmp/siglongjmp trick (setjmp/longjmp on platforms without the "sig" versions) is to sigsetjmp() before select(), poll() or other syscall, then set a jump-enable flag (or just jmp_buf pointer) telling your signal handlers to siglongjmp(), then clear the flag after the select() etc syscall returns. This is more complicated than self-pipe in a number of ways (and a bit slower), because you have to be careful to prevent nested signal handlers or deal with them appropriately, and it's doesn't work reliably on all platforms. So I would stick with the self-pipe trick.
Unklejoe 5y ago
> No new code should be using select(). Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. What should we use instead? poll()?
- johnny22 5y ago
  that's indeed one of the choices from the article itself.
  - Unklejoe 5y ago
    I remember reading somewhere that poll was just a wrapper for select. I’m guessing that’s either outdated or false now.
    - cesarb 5y ago
      It's correct but outdated: according to https://man7.org/linux/man-pages/man2/poll.2.html#VERSIONS https://man7.org/linux/man-pages/man2/poll.2.html#VERSIONS "The poll() system call was introduced in Linux 2.1.23. On older kernels that lack this system call, the glibc poll() wrapper function provides emulation using select(2)." So as long as you use a Linux kernel made this century, you will be fine.
- friend-monoid 5y ago
  epoll.
cesarb 5y ago
> No new code should be using select(). Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. Perhaps it might be time to start treating select() like gets() was treated: the compiler should emit a warning by default whenever it's used, and after perhaps a decade, the API should no longer be available for newly compiled code. While not as bad as gets(), which according to Rusty's classification of API quality (https://ozlabs.org/~rusty/index.cgi/tech/2008-03-30.html https://ozlabs.org/~rusty/index.cgi/tech/2008-03-30.html and https://ozlabs.org/~rusty/index.cgi/tech/2008-04-01.html https://ozlabs.org/~rusty/index.cgi/tech/2008-04-01.html) gets a -10 ("It's impossible to get right"), select() is around a -5 ("Do it right and it will sometimes break at runtime"), or perhaps even a -7 ("The obvious use is wrong").
- pengaru 5y ago
  I agree 100% re: compiler throwing a warning like gets() for select().
- deleted 5y ago
  [deleted]
- viraptor 5y ago
  This makes me wonder if we could realistically start enforcing SystemCallFilter on new services which immediately crashes an app of it tries to use select(). I suspect there's a lot of them that would fail at the moment... But maybe in a few years.
- kevin_thibedeau 5y ago
  select() is useful on platforms without any poll variants.
  - bonzini 5y ago
    Of which the only one left is Windows, where you probably have a completely different event loop anyway. The Windows select by the way does not have the limitations on the file descriptor that the article talks about, because Windows file descriptors are not small consecutive integers.
moises_silva 5y ago
> No new code should be using select(). Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. I'm with you here. I'm also guilty of using it still for some CLI tools or other small programs. Just to be clear you don't need to assume all fd values will stay within FD_SETSIZE. Just that the fds you will use with select will stay in that range. Even if there will be lots of fds (more than FD_SETSIZE) for other resources, it works for example to select on stdin if you're not doing funny business reopening it. There's no much of an excuse to keep using it though other than laziness.
- bvrmn 5y ago
  Laziness is pretty good reason to continue to use select. It's a convenient interface for simple tasks. The very nature of fire a set of fds and get a result beats (e)poll/kqueue/fd-registering based API in UX regard. It's sad there is no alternative with same simple interface and without FD_SETSIZE shortcoming. Edit: I'm biased by Python which has `select.poll` with API similar to epoll and it's really hard to use it instead of select.
  - megous 5y ago
    poll() is even easier to use, imo.
wiml 5y ago
> People tend to forget that it's the value of the fds you pass to select() which can't exceed FD_SETSIZE Is that true? I have a memory of working on systems where you can pass larger arrays in if you want, you just have to allocate your own bitvectors instead of using the convenient struct. A quick look at the linux kernel makes me think that this is true for Linux; the kernel doesn't appear to reference FD_SETSIZE in its implementation.
- TheDong 5y ago
  If you read the man page bugs section: https://man7.org/linux/man-pages/man2/select.2.html https://man7.org/linux/man-pages/man2/select.2.html "The Linux kernel imposes no fixed limit, but the glibc implementation makes fd_set a fixed-size type, with FD_SETSIZE defined as 1024" Everyone using select, with very few exceptions, is using glibc. That limitation does appear to exist in glibc if you look at the source code. The source code requires you to pass a glibc defined 'fd_set' type https://man7.org/linux/man-pages/man2/select.2.html https://man7.org/linux/man-pages/man2/select.2.html That struct is defined here: https://github.com/bminor/glibc/blob/595c22ecd8e87a27fd19270ed30fdbae9ad25426/misc/sys/select.h#L58-L70 https://github.com/bminor/glibc/blob/595c22ecd8e87a27fd19270... It does seem to have the limits described by the blogpost.
  - ksherlock 5y ago
    The glibc/musl select library call doesn't verify nfds is < FD_SETSIZE so you could create your own fd_set as large as you want it. glibc: https://github.com/bminor/glibc/blob/master/sysdeps/unix/sysv/linux/select.c https://github.com/bminor/glibc/blob/master/sysdeps/unix/sys... musl: https://git.musl-libc.org/cgit/musl/tree/src/select/select.c https://git.musl-libc.org/cgit/musl/tree/src/select/select.c *BSD allows you to define your own FD_SETSIZE before including system headers.
  - toast0 5y ago
    On FreeBSD's select(2) man page, the NOTES section has the much friendlier > The default size of FD_SETSIZE is currently 1024. In order to accommodate programs which might potentially use a larger number of open files with select(), it is possible to increase this size by having the program define FD_SETSIZE before the inclusion of any header which includes <sys/types.h>. Although, if you link to a library which uses select and doesn't check that fds it uses are valid with the FD_SETSIZE it was compiled with, you're still in for a bad time, as suggested in the article. And still, probably use something better than select.
anonymousiam 5y ago
Maybe I am out of date. (I probably am because I haven't written any code that uses select() for two decades.) Can somebody please identify the POSIX method by which one can wait on a set of descriptors, that does not involve spinning in a polling loop and gobbling up a CPU thread? The last time I had a need for such a thing, select() was the preferred way to do that.
- DDR0 5y ago
  The article mentions a few near the end, I think.
  - masklinn 5y ago
    The article mentions one (poll) and doesn’t make it clear they it’s posix. The rest is Linux specific. Aside from being a lot more recent (posix 2001), poll was also broken on macOS until 10.9. Though you can always use kqueue instead that remains a portability annoyance: Linux still doesn’t have kqueue.
    - yxhuvud 5y ago
      > still doesn't have kqueue And with io_uring around it never will unless someone does a pure userspace implementation. Not certain if possible but it might be.
      - Others 5y ago
        Like: https://github.com/mheily/libkqueue https://github.com/mheily/libkqueue ?
- sgtnoodle 5y ago
  Epoll works really well in Linux.
  - anonymousiam 5y ago
    I'm sure that epoll() works great, but I was asking about a POSIX replacement. Some people still write portable code.
dataflow 5y ago
n.b. FD_SETSIZE on Windows represents the quantity (!)
IshKebab 5y ago
> No new code should be using select(). Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. That's what most of the article is about.
geocar 5y ago
> No new code should be using select(). It’s faster than poll, and if you are accept()ing a lot, faster than epoll. Multiple threads can do better, but only by working together. SO_REUSEPORT+select is faster if you can get away with it. Iouring looks promising though. > Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. That’s what the rlimit is for. If you call a program that uses select with 1025 open FD you get what you deserve.
- lelanthran 5y ago
  >> Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. > >That’s what the rlimit is for. If you call a program that uses select with 1025 open FD you get what you deserve. I thought that FD_SETSIZE was a compile-time constant that limited what values for file descriptors could be passed to select. IOW, it doesn't matter if you have only two files open; if one of them has an fd with a value of 1026, then select will do the wrong thing.
  - justincormack 5y ago
    Yeah but fds are guaranteed to be opened lowest available on first, so thats ok, you cant get fd=1026 without having 1026 open. The Linux syscall does not have a fixed FD_SETSIZE so glibc could increase the size.
    - geocar 5y ago
      > Yeah but fds are guaranteed to be opened lowest available on first, so thats ok, you cant get fd=1026 without having 1026 open. This is exactly correct. > The Linux syscall does not have a fixed FD_SETSIZE so glibc could increase the size. The application writer can define FD_SETSIZE before including <sys/socket.h> and set it to whatever they want on BSD (including OSX). glibc could* adopt this trick, but my feeling is you probably don't want to stack-allocate this stuff anyway. Dealing with a large number of very-active connections concurrently efficiently requires more effort than just this though.
    - 5y ago

ridiculous_fish 5y ago

Is there a portable alternative to pselect()? There's ppoll(), but it's missing on some platforms.

wahern 5y ago
At this point I believe ppoll has been added to every mainstream Unix except macOS. It's also been accepted for the next POSIX revision. For many years pselect on macOS didn't actually fix the race condition, notwithstanding its UNIX03 certification. pselect was just a naive wrapper around select that didn't atomically unmask signals. So I ended up having to create a kqueue-based implementation for use on macOS and stragglers like OpenBSD which at that time lacked pselect entirely. See https://github.com/wahern/cqueues/blob/87f14f9/src/cqueues.c#L2489 https://github.com/wahern/cqueues/blob/87f14f9/src/cqueues.c... I can't remember if pselect has been fixed on macOS. The same approach could be used to emulate ppoll on macOS. Though epoll, kqueue, and Solaris ports are so similar that you can easily wrap all of them behind a thin shim. See https://github.com/wahern/cqueues/blob/87f14f9/src/lib/kpoll.c https://github.com/wahern/cqueues/blob/87f14f9/src/lib/kpoll... (Caution: it's an incomplete refactor of a similar shim I used in some non-open source code. The original code is even simpler and has seen extensive use. The refactored shim should work but is untested.) AIX is really the only mainstream Unix lacking an API w/ equivalent semantics. AIX has a better-than-poll interface, but it doesn't work anything like the others. Notably, epoll, kqueue, and Solaris ports control descriptors are all pollable in their own right, permitting you to nest different [readiness polling-based] event loop frameworks.
jlokier 5y ago
There is a way to simulate it portably, but it's a little complicated. You can use the self-pipe trick portably on POSIX systems I believe. (There is also the sigsetjmp/siglongjmp trick on most platforms, but I remember it breaking on some version of Cygwin.) The self-pipe trick is where you create a pipe(), add the read side to select(), poll() or other syscall, and have your signal handlers write a byte to the write side. When reading the pipe to flush this state, remember to read until it's empty because there may have been multiple signals. Reading 2 bytes and looping if you get 2 will get this done usually in one syscall. It's best in principle to make the pipe non-blocking so the writes can never block if the pipe fills. This is basically the portable equivalent to Linux's eventfd. (If you're worried about out-of-memory preventing non-blocking write of a single byte, you can close the pipe instead, but this causes several complications because it's not safe to close the same fd twice, and you may have signals interrupt other signal handlers, so race hazards around closing the fd twice must be protected against by blocking signals inside the signal handler and using more flags.) You can optimise self-pipe by setting a write-enable flag just before select() etc and clearing it afterwards, with the signal handlers only writing a byte (or closing the pipe) when the flag is set, and setting a separate flag to say that the signal occurred, which is checked between setting the write-enable flag and before the select() etc syscall. Check the signal-happened flag after select() etc and if it's set, read the pipe to flush even if it doesn't show as readable in the fd set, because the signal could have happened later. A further optimisation is for the signal handler to clear write-enable when writing the pipe, so subsequent signals don't do it again. Don't assume this means there can only be one byte in the pipe, because nested signals introduce a race hazard to writing and clearing write-enable atomically. The sigsetjmp/siglongjmp trick (setjmp/longjmp on platforms without the "sig" versions) is to sigsetjmp() before select(), poll() or other syscall, then set a jump-enable flag (or just jmp_buf pointer) telling your signal handlers to siglongjmp(), then clear the flag after the select() etc syscall returns. This is more complicated than self-pipe in a number of ways (and a bit slower), because you have to be careful to prevent nested signal handlers or deal with them appropriately, and it's doesn't work reliably on all platforms. So I would stick with the self-pipe trick.

Unklejoe 5y ago

> No new code should be using select(). Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. What should we use instead? poll()?

johnny22 5y ago
that's indeed one of the choices from the article itself.
- Unklejoe 5y ago
  I remember reading somewhere that poll was just a wrapper for select. I’m guessing that’s either outdated or false now.
  - cesarb 5y ago
    It's correct but outdated: according to https://man7.org/linux/man-pages/man2/poll.2.html#VERSIONS https://man7.org/linux/man-pages/man2/poll.2.html#VERSIONS "The poll() system call was introduced in Linux 2.1.23. On older kernels that lack this system call, the glibc poll() wrapper function provides emulation using select(2)." So as long as you use a Linux kernel made this century, you will be fine.
friend-monoid 5y ago
epoll.

cesarb 5y ago

> No new code should be using select(). Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. Perhaps it might be time to start treating select() like gets() was treated: the compiler should emit a warning by default whenever it's used, and after perhaps a decade, the API should no longer be available for newly compiled code. While not as bad as gets(), which according to Rusty's classification of API quality (https://ozlabs.org/~rusty/index.cgi/tech/2008-03-30.html https://ozlabs.org/~rusty/index.cgi/tech/2008-03-30.html and https://ozlabs.org/~rusty/index.cgi/tech/2008-04-01.html https://ozlabs.org/~rusty/index.cgi/tech/2008-04-01.html) gets a -10 ("It's impossible to get right"), select() is around a -5 ("Do it right and it will sometimes break at runtime"), or perhaps even a -7 ("The obvious use is wrong").

pengaru 5y ago
I agree 100% re: compiler throwing a warning like gets() for select().
deleted 5y ago
[deleted]
viraptor 5y ago
This makes me wonder if we could realistically start enforcing SystemCallFilter on new services which immediately crashes an app of it tries to use select(). I suspect there's a lot of them that would fail at the moment... But maybe in a few years.
kevin_thibedeau 5y ago
select() is useful on platforms without any poll variants.
- bonzini 5y ago
  Of which the only one left is Windows, where you probably have a completely different event loop anyway. The Windows select by the way does not have the limitations on the file descriptor that the article talks about, because Windows file descriptors are not small consecutive integers.

moises_silva 5y ago

> No new code should be using select(). Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. I'm with you here. I'm also guilty of using it still for some CLI tools or other small programs. Just to be clear you don't need to assume all fd values will stay within FD_SETSIZE. Just that the fds you will use with select will stay in that range. Even if there will be lots of fds (more than FD_SETSIZE) for other resources, it works for example to select on stdin if you're not doing funny business reopening it. There's no much of an excuse to keep using it though other than laziness.

bvrmn 5y ago
Laziness is pretty good reason to continue to use select. It's a convenient interface for simple tasks. The very nature of fire a set of fds and get a result beats (e)poll/kqueue/fd-registering based API in UX regard. It's sad there is no alternative with same simple interface and without FD_SETSIZE shortcoming. Edit: I'm biased by Python which has `select.poll` with API similar to epoll and it's really hard to use it instead of select.
- megous 5y ago
  poll() is even easier to use, imo.

wiml 5y ago

> People tend to forget that it's the value of the fds you pass to select() which can't exceed FD_SETSIZE Is that true? I have a memory of working on systems where you can pass larger arrays in if you want, you just have to allocate your own bitvectors instead of using the convenient struct. A quick look at the linux kernel makes me think that this is true for Linux; the kernel doesn't appear to reference FD_SETSIZE in its implementation.

TheDong 5y ago
If you read the man page bugs section: https://man7.org/linux/man-pages/man2/select.2.html https://man7.org/linux/man-pages/man2/select.2.html "The Linux kernel imposes no fixed limit, but the glibc implementation makes fd_set a fixed-size type, with FD_SETSIZE defined as 1024" Everyone using select, with very few exceptions, is using glibc. That limitation does appear to exist in glibc if you look at the source code. The source code requires you to pass a glibc defined 'fd_set' type https://man7.org/linux/man-pages/man2/select.2.html https://man7.org/linux/man-pages/man2/select.2.html That struct is defined here: https://github.com/bminor/glibc/blob/595c22ecd8e87a27fd19270ed30fdbae9ad25426/misc/sys/select.h#L58-L70 https://github.com/bminor/glibc/blob/595c22ecd8e87a27fd19270... It does seem to have the limits described by the blogpost.
- ksherlock 5y ago
  The glibc/musl select library call doesn't verify nfds is < FD_SETSIZE so you could create your own fd_set as large as you want it. glibc: https://github.com/bminor/glibc/blob/master/sysdeps/unix/sysv/linux/select.c https://github.com/bminor/glibc/blob/master/sysdeps/unix/sys... musl: https://git.musl-libc.org/cgit/musl/tree/src/select/select.c https://git.musl-libc.org/cgit/musl/tree/src/select/select.c *BSD allows you to define your own FD_SETSIZE before including system headers.
- toast0 5y ago
  On FreeBSD's select(2) man page, the NOTES section has the much friendlier > The default size of FD_SETSIZE is currently 1024. In order to accommodate programs which might potentially use a larger number of open files with select(), it is possible to increase this size by having the program define FD_SETSIZE before the inclusion of any header which includes <sys/types.h>. Although, if you link to a library which uses select and doesn't check that fds it uses are valid with the FD_SETSIZE it was compiled with, you're still in for a bad time, as suggested in the article. And still, probably use something better than select.

anonymousiam 5y ago

Maybe I am out of date. (I probably am because I haven't written any code that uses select() for two decades.) Can somebody please identify the POSIX method by which one can wait on a set of descriptors, that does not involve spinning in a polling loop and gobbling up a CPU thread? The last time I had a need for such a thing, select() was the preferred way to do that.

DDR0 5y ago
The article mentions a few near the end, I think.
- masklinn 5y ago
  The article mentions one (poll) and doesn’t make it clear they it’s posix. The rest is Linux specific. Aside from being a lot more recent (posix 2001), poll was also broken on macOS until 10.9. Though you can always use kqueue instead that remains a portability annoyance: Linux still doesn’t have kqueue.
  - yxhuvud 5y ago
    > still doesn't have kqueue And with io_uring around it never will unless someone does a pure userspace implementation. Not certain if possible but it might be.
    - Others 5y ago
      Like: https://github.com/mheily/libkqueue https://github.com/mheily/libkqueue ?
sgtnoodle 5y ago
Epoll works really well in Linux.
- anonymousiam 5y ago
  I'm sure that epoll() works great, but I was asking about a POSIX replacement. Some people still write portable code.

dataflow 5y ago

n.b. FD_SETSIZE on Windows represents the quantity (!)

IshKebab 5y ago

> No new code should be using select(). Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. That's what most of the article is about.

geocar 5y ago

> No new code should be using select(). It’s faster than poll, and if you are accept()ing a lot, faster than epoll. Multiple threads can do better, but only by working together. SO_REUSEPORT+select is faster if you can get away with it. Iouring looks promising though. > Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. That’s what the rlimit is for. If you call a program that uses select with 1025 open FD you get what you deserve.

lelanthran 5y ago
>> Using it assumes the execution environment keeps all file descriptor values within FD_SETSIZE. > >That’s what the rlimit is for. If you call a program that uses select with 1025 open FD you get what you deserve. I thought that FD_SETSIZE was a compile-time constant that limited what values for file descriptors could be passed to select. IOW, it doesn't matter if you have only two files open; if one of them has an fd with a value of 1026, then select will do the wrong thing.
- justincormack 5y ago
  Yeah but fds are guaranteed to be opened lowest available on first, so thats ok, you cant get fd=1026 without having 1026 open. The Linux syscall does not have a fixed FD_SETSIZE so glibc could increase the size.
  - geocar 5y ago
    > Yeah but fds are guaranteed to be opened lowest available on first, so thats ok, you cant get fd=1026 without having 1026 open. This is exactly correct. > The Linux syscall does not have a fixed FD_SETSIZE so glibc could increase the size. The application writer can define FD_SETSIZE before including <sys/socket.h> and set it to whatever they want on BSD (including OSX). glibc could* adopt this trick, but my feeling is you probably don't want to stack-allocate this stuff anyway. Dealing with a large number of very-active connections concurrently efficiently requires more effort than just this though.
  - 5y ago