5 ms·

The real fun thing is when the same application is using “select()” and then somewhere else you open like 5000 files. Then you start getting weird crashes and e

The real fun thing is when the same application is using “select()” and then somewhere else you open like 5000 files. Then you start getting weird crashes and eventually trace it down to the select bitset having a hardcoded max of 4096 entries and no bounds checking! Fun fun fun.

danadam 1y ago
> trace it down to the select bitset having a hardcoded max of 4096 Did it change? Last time I checked it was 1024 (though it was long time ago). > and no bounds checking! _FORTIFY_SOURCE is not set? When I try to pass 1024 to FD_SET and FD_CLR on my (very old) machine I immediately get: *** buffer overflow detected ***: ./a.out terminated Aborted (ok, with -O1 and higher)
- xorvoid 1y ago
  You’re right. I think it ends up working out to a 4096 page on x86 machines, that’s probably what I remembered. Yes, _FORTIFY_SOURCE is a fabulous idea. I was just a bit shocked it wasn’t checked without _FORTIFY_SOURCE. If you’re doing FD_SET/FD_CLR, you’re about to make an (expensive) syscall. Why do you care to elide a cheap not-taken branch that’ll save your bacon some day? The overhead is so incredibly negligible. Anyways, seriously just use poll(). The select() syscall needs to go away for good.
  - reisse 1y ago
    You've had a good chance to really see 4096 descriptions in select() somewhere. The man is misleading because it refers to the stubbornly POSIX compliant glibc wrapper around actual syscall. Any sane modern kernel (Linux; FreeBSD; NT (although select() on NT is a very different beast); well, maybe except macOS, never had a chance to write network code there) supports passing the descriptor sets of arbitrary size to select(). It's mentioned further down in the man, in the BUGS section: > POSIX allows an implementation to define an upper limit, advertised via the constant FD_SETSIZE, on the range of file descriptors that can be specified in a file descriptor set. The Linux kernel imposes no fixed limit, but the glibc implementation makes fd_set a fixed-size type, with FD_SETSIZE defined as 1024, and the FD_*() macros operating according to that limit. The code I've had a chance to work with (it had its roots in the 90s-00s, therefore the select()) mostly used 2048 and 4096. > Anyways, seriously just use poll(). Oh please don't. poll() should be in the same grave as select() really. Either use libev/libuv or go down the rabbit hole of what is the bleeding edge IO multiplexer for your platform (kqueue/epoll/IOCP/io_uring...).
moyix 1y ago
I made a CTF challenge based on that lovely feature of select() :D You could use the out-of-bounds bitset memory corruption to flip bits in an RSA public key in a way that made it factorable, generate the corresponding private key, and use that to authenticate. https://threadreaderapp.com/thread/1723398619313603068.html https://threadreaderapp.com/thread/1723398619313603068.html
- malux85 1y ago
  Oh that’s clever!
- saagarjha 1y ago
  [flagged]
- StefanBatory 1y ago
  I love how you've made it Eva themed, my respect to you.
ape4 1y ago
Yeah, the man page says: WARNING: select() can monitor only file descriptors numbers that are less than FD_SETSIZE (1024)—an unreasonably low limit for many modern applications—and this limitation will not change. All modern applica‐ tions should instead use poll(2) or epoll(7), which do not suffer this limitation.
- time4tea 1y ago
  You can recompile libc if you want to change the limit, or at least you could in the past.
  - o11c 1y ago
    You don't have to recompile, just do the following (at least on glibc): #include <sys/types.h> // pull in initial definition of __FD_SETSIZE #undef __FD_SETSIZE #define __FD_SETSIZE 32768 // or whatever #include <sys/select.h> // won't include the internal <bits/types.h> again This is a rare case when `-Wsystem-headers` is useful to enable (and these days system headers are usually pretty clean) - it will catch if you accidentally define `__FD_SETSIZE` before the system does. Note that `select` is still the nicest API in a lot of ways - `poll` wastes space gratuitously, `epoll` requires lots of finicky `modify` syscalls, and `io_uring` is frankly not sane. That said: * if you're only dealing with a couple FDs, use `poll`. * it's not that hard to take a day and think about epoll write buffer management. You need to consider every combination of: epoll state is/isn't checking writability (you want to only change this lazily) on the previous/current iteration, was there nothing/something in the write buffer? prior actual write was would-block/actually-incomplete/spuriously-incomplete/complete current actual write ends up would-block/actually-incomplete/spuriously-incomplete/complete There are many "correct" answers, but I suspect the optimal answer for epoll is something like: initially, write optimistically (and do this before the wait). If you fail to write anything at all, enable the kernel flag. For FDs that you've previously enabled the flag for, if you don't have anything to write this time, disable the flag; otherwise, don't actually write until after the wait (it is guaranteed to return immediately if the write would be allowed, after all, but you'll also get other events that happen to be ready). If you trust your event handlers to return quickly, you can defer any indicated writes until the next wait, otherwise do them before handling events. You can see why people still use `select`.
reisse 1y ago
Oh the real fun thing is when the select() is not even in your code! I remember having to integrate a closed-source third-party library vendored by an Australian fin(tech?) company which used select() internally, into a bigger application which really liked to open a lot of file descriptors. Their devs refused to rewrite it to use something more contemporary (it was 2019 iirc!), so we had to improvise. In the end we came up with a hack to open 4k file descriptors into /dev/null on start, then open the real files and sockets necessary for our app, then close that /dev/null descriptors and initialize the library.
- o11c 1y ago
  There's no need to actually do all the opening if you control the code. You can do anything with `fcntl(F_DUPFD{,_CLOEXEC})` and `fdopen`.
  - reisse 1y ago
    If we had control of library code, we'd just get rid of select()... Though we did use the dup trick in another case!
cryptonector 1y ago
Or back in the days of Solaris 9 and under, 32-bit processes could not have stdio handles with file descriptor numbers larger than 255. Super double plus unfun when you got hit by that. Remember that, u/lukeh?

danadam 1y ago

> trace it down to the select bitset having a hardcoded max of 4096 Did it change? Last time I checked it was 1024 (though it was long time ago). > and no bounds checking! _FORTIFY_SOURCE is not set? When I try to pass 1024 to FD_SET and FD_CLR on my (very old) machine I immediately get: *** buffer overflow detected ***: ./a.out terminated Aborted (ok, with -O1 and higher)

xorvoid 1y ago
You’re right. I think it ends up working out to a 4096 page on x86 machines, that’s probably what I remembered. Yes, _FORTIFY_SOURCE is a fabulous idea. I was just a bit shocked it wasn’t checked without _FORTIFY_SOURCE. If you’re doing FD_SET/FD_CLR, you’re about to make an (expensive) syscall. Why do you care to elide a cheap not-taken branch that’ll save your bacon some day? The overhead is so incredibly negligible. Anyways, seriously just use poll(). The select() syscall needs to go away for good.
- reisse 1y ago
  You've had a good chance to really see 4096 descriptions in select() somewhere. The man is misleading because it refers to the stubbornly POSIX compliant glibc wrapper around actual syscall. Any sane modern kernel (Linux; FreeBSD; NT (although select() on NT is a very different beast); well, maybe except macOS, never had a chance to write network code there) supports passing the descriptor sets of arbitrary size to select(). It's mentioned further down in the man, in the BUGS section: > POSIX allows an implementation to define an upper limit, advertised via the constant FD_SETSIZE, on the range of file descriptors that can be specified in a file descriptor set. The Linux kernel imposes no fixed limit, but the glibc implementation makes fd_set a fixed-size type, with FD_SETSIZE defined as 1024, and the FD_*() macros operating according to that limit. The code I've had a chance to work with (it had its roots in the 90s-00s, therefore the select()) mostly used 2048 and 4096. > Anyways, seriously just use poll(). Oh please don't. poll() should be in the same grave as select() really. Either use libev/libuv or go down the rabbit hole of what is the bleeding edge IO multiplexer for your platform (kqueue/epoll/IOCP/io_uring...).

moyix 1y ago

I made a CTF challenge based on that lovely feature of select() :D You could use the out-of-bounds bitset memory corruption to flip bits in an RSA public key in a way that made it factorable, generate the corresponding private key, and use that to authenticate. https://threadreaderapp.com/thread/1723398619313603068.html https://threadreaderapp.com/thread/1723398619313603068.html

malux85 1y ago
Oh that’s clever!
saagarjha 1y ago
[flagged]
StefanBatory 1y ago
I love how you've made it Eva themed, my respect to you.

ape4 1y ago

Yeah, the man page says: WARNING: select() can monitor only file descriptors numbers that are less than FD_SETSIZE (1024)—an unreasonably low limit for many modern applications—and this limitation will not change. All modern applica‐ tions should instead use poll(2) or epoll(7), which do not suffer this limitation.

time4tea 1y ago
You can recompile libc if you want to change the limit, or at least you could in the past.
- o11c 1y ago
  You don't have to recompile, just do the following (at least on glibc): #include <sys/types.h> // pull in initial definition of __FD_SETSIZE #undef __FD_SETSIZE #define __FD_SETSIZE 32768 // or whatever #include <sys/select.h> // won't include the internal <bits/types.h> again This is a rare case when `-Wsystem-headers` is useful to enable (and these days system headers are usually pretty clean) - it will catch if you accidentally define `__FD_SETSIZE` before the system does. Note that `select` is still the nicest API in a lot of ways - `poll` wastes space gratuitously, `epoll` requires lots of finicky `modify` syscalls, and `io_uring` is frankly not sane. That said: * if you're only dealing with a couple FDs, use `poll`. * it's not that hard to take a day and think about epoll write buffer management. You need to consider every combination of: epoll state is/isn't checking writability (you want to only change this lazily) on the previous/current iteration, was there nothing/something in the write buffer? prior actual write was would-block/actually-incomplete/spuriously-incomplete/complete current actual write ends up would-block/actually-incomplete/spuriously-incomplete/complete There are many "correct" answers, but I suspect the optimal answer for epoll is something like: initially, write optimistically (and do this before the wait). If you fail to write anything at all, enable the kernel flag. For FDs that you've previously enabled the flag for, if you don't have anything to write this time, disable the flag; otherwise, don't actually write until after the wait (it is guaranteed to return immediately if the write would be allowed, after all, but you'll also get other events that happen to be ready). If you trust your event handlers to return quickly, you can defer any indicated writes until the next wait, otherwise do them before handling events. You can see why people still use `select`.

reisse 1y ago

Oh the real fun thing is when the select() is not even in your code! I remember having to integrate a closed-source third-party library vendored by an Australian fin(tech?) company which used select() internally, into a bigger application which really liked to open a lot of file descriptors. Their devs refused to rewrite it to use something more contemporary (it was 2019 iirc!), so we had to improvise. In the end we came up with a hack to open 4k file descriptors into /dev/null on start, then open the real files and sockets necessary for our app, then close that /dev/null descriptors and initialize the library.

o11c 1y ago
There's no need to actually do all the opening if you control the code. You can do anything with `fcntl(F_DUPFD{,_CLOEXEC})` and `fdopen`.
- reisse 1y ago
  If we had control of library code, we'd just get rid of select()... Though we did use the dup trick in another case!

cryptonector 1y ago

Or back in the days of Solaris 9 and under, 32-bit processes could not have stdio handles with file descriptor numbers larger than 255. Super double plus unfun when you got hit by that. Remember that, u/lukeh?