3 ms·

Not informed enough to say if the code is as interesting as the title implies. Hope someone comments to enlighten me, shall upvote if so!

by corin_ 16y ago

rwmj 16y ago
You can send a file descriptor over a socket between two processes. (Google "SCM_RIGHTS"). This is one place in the kernel where it employs a garbage collector (hey, they need to start somewhere ...). The reason is that there's an uncomfortable part about this API: the reader doesn't necessarily have to pick up the file descriptor it is sent. If the sender closes the fd, the kernel needs to detect that the reader didn't pick up the fd and deallocate the underlying file resource. Hence GC needed. This code makes sure that many such fds are "in flight" over the socketpair, and by closing the socket instead of reading from it, ensures they are leaked. The GC doesn't get invoked under this particular set of circumstances so eventually you consume all system file resources. (I think even ulimit doesn't stop this). Solution as in the follow up message is to run the GC in an extra place.
- JoachimSchipper 16y ago
  Well, apparently the GC doesn't work too well here (the self-followup talks about a non-maskable interrupt watchdog triggering, i.e. the GC takes too long). But yes, I don't understand why this would necessarily be unsolvable.
- tbrownaw 16y ago
  > and by closing the socket instead of reading from it, ensures they are leaked. It does: create socketpair N+1 send socketpair N over socketpair N+1 close socketpair N repeat So the only ones that are ever closed are the local copy of something that was sent over another socketpair. So all of the fd's are reachable, assuming that an in-flight fd is (like a received fd) a copy of the original fd rather than a reference to it.
  - caf 16y ago
    Seems like in-flight fds should still count against the sending process's open file limit. (So that all fds are counted against some processes open file limit).

Not informed enough to say if the code is as interesting as the title implies. Hope someone comments to enlighten me, shall upvote if so!

by corin_ 16y ago

Not informed enough to say if the code is as interesting as the title implies. Hope someone comments to enlighten me, shall upvote if so!

rwmj 16y ago

You can send a file descriptor over a socket between two processes. (Google "SCM_RIGHTS"). This is one place in the kernel where it employs a garbage collector (hey, they need to start somewhere ...). The reason is that there's an uncomfortable part about this API: the reader doesn't necessarily have to pick up the file descriptor it is sent. If the sender closes the fd, the kernel needs to detect that the reader didn't pick up the fd and deallocate the underlying file resource. Hence GC needed. This code makes sure that many such fds are "in flight" over the socketpair, and by closing the socket instead of reading from it, ensures they are leaked. The GC doesn't get invoked under this particular set of circumstances so eventually you consume all system file resources. (I think even ulimit doesn't stop this). Solution as in the follow up message is to run the GC in an extra place.

JoachimSchipper 16y ago
Well, apparently the GC doesn't work too well here (the self-followup talks about a non-maskable interrupt watchdog triggering, i.e. the GC takes too long). But yes, I don't understand why this would necessarily be unsolvable.
tbrownaw 16y ago
> and by closing the socket instead of reading from it, ensures they are leaked. It does: create socketpair N+1 send socketpair N over socketpair N+1 close socketpair N repeat So the only ones that are ever closed are the local copy of something that was sent over another socketpair. So all of the fd's are reachable, assuming that an in-flight fd is (like a received fd) a copy of the original fd rather than a reference to it.
- caf 16y ago
  Seems like in-flight fds should still count against the sending process's open file limit. (So that all fds are counted against some processes open file limit).