4 ms·

When another thread pushes a socket to be processed by the socket polling thread, the socket polling thread calls select() on that function. Since select() is a

by rsp1984 5y ago

When another thread pushes a socket to be processed by the socket polling thread, the socket polling thread calls select() on that function. Since select() is a blocking call, when another socket is pushed to the socket polling thread queue it has to somehow interrupt select() so the new socket gets processed ASAP. How does one interrupt select() function? Apparently, we used QueueUserAPC() to execute asynchronous procedure while select() was blocked… and threw an exception out of it! Not taking anything away from the great debugging, but that is just incredibly bad design and reeks of bugs from a mile away. System calls block for a reason. If you can't handle the blocking, how about a non-blocking architecture? But in this case, the engineer thought "hey, I don't want to block on my precious main thread, so let's just give this blocking thing to another thread, that'll surely solve it!". Except it doesn't when you use that same thread as a worker to execute all other thread's blocking crap ... but then write the queue push() function to be blocking as well! Keep it simple folks. Bad design always breeds more bad design.

oshiar53-0 5y ago
https://www.youtube.com/watch?v=k238XpMMn38 https://www.youtube.com/watch?v=k238XpMMn38 but it's the rapidly dwindling sanity of unity programmers
syntheticnature 5y ago
It's particularly surprising to me because the whole point of select() is to handle these sorts of cases easily. The technique I've often seen is just to have a pipe or other local connection to the thread using select, and then you wake up select when you need it to add something to the queue.
- aftbit 5y ago
  That was their fix.
  - syntheticnature 5y ago
    My over-editing of my comment got rid of the sense of "they knew it was the fix, why not do that in the first place?" Which, admittedly, sometimes folks just don't know, but usually I more often run into complete cluelesness that select()/poll() and friends exist.
jstimpfle 5y ago
I'd go for a different approach: Add some cancellation event for select() to listen to as well. Then, when another thread submits a new event source, it's only a matter of signalling the cancellation event, so the select() thread can update its poll set. "Non-blocking" sounds very special but for anything except batch programs it should be used by default because whenever there is more than one possible input (inputs can be trivial as a cancellation signal) one can not afford to be blocked. Then again, that doesn't mean that one shouldn't use select(). Unless one goes ultra-high frequency where the CPU is busy almost 100% of the time, select() saves CPU cycles, and decreases response time. Without select(), finding an appropriate polling frequency is a tradeoff between wasted compute cycles and increased response time.
- zbentley 5y ago
  > I'd go for a different approach: Add some cancellation event for select() to listen to as well. That's exactly what the author of the article did.
  - deleted 5y ago
    [deleted]
robflynn 5y ago
I was once brought over to a project that used exceptions for flow control, sometimes nested several layers deep. It was a nightmare to debug and untangle.
- akira2501 5y ago
  What's funny to me is their solution is very natural and commonly used in unix. My guess is because signal handling on unix is much more prevalent and brings with it many of the same control flow issues you'd see in exceptions.
Matthias247 5y ago
There's nothing super special about this design. It apparently uses a single "IO reactor" which handles readiness notifications for a couple of other threads. Those might be threadpool, and/or run non-blocking code (we don't know too much about this). This design is pretty much the same that you can e.g. find in the go runtime (netpoll), the .NET core runtime, Rust multithreaded async libraries, Java nio2, etc. The only thing that was special here is that they used an APC for waking up the reactor in case new events had to be added. Its non-standard, and kind of comparable to using a signal on unix to interrupt `select` instead of a message on a preregistered socket.

oshiar53-0 5y ago

https://www.youtube.com/watch?v=k238XpMMn38 https://www.youtube.com/watch?v=k238XpMMn38 but it's the rapidly dwindling sanity of unity programmers

syntheticnature 5y ago

It's particularly surprising to me because the whole point of select() is to handle these sorts of cases easily. The technique I've often seen is just to have a pipe or other local connection to the thread using select, and then you wake up select when you need it to add something to the queue.

aftbit 5y ago
That was their fix.
- syntheticnature 5y ago
  My over-editing of my comment got rid of the sense of "they knew it was the fix, why not do that in the first place?" Which, admittedly, sometimes folks just don't know, but usually I more often run into complete cluelesness that select()/poll() and friends exist.

jstimpfle 5y ago

I'd go for a different approach: Add some cancellation event for select() to listen to as well. Then, when another thread submits a new event source, it's only a matter of signalling the cancellation event, so the select() thread can update its poll set. "Non-blocking" sounds very special but for anything except batch programs it should be used by default because whenever there is more than one possible input (inputs can be trivial as a cancellation signal) one can not afford to be blocked. Then again, that doesn't mean that one shouldn't use select(). Unless one goes ultra-high frequency where the CPU is busy almost 100% of the time, select() saves CPU cycles, and decreases response time. Without select(), finding an appropriate polling frequency is a tradeoff between wasted compute cycles and increased response time.

zbentley 5y ago
> I'd go for a different approach: Add some cancellation event for select() to listen to as well. That's exactly what the author of the article did.
- deleted 5y ago
  [deleted]

robflynn 5y ago

I was once brought over to a project that used exceptions for flow control, sometimes nested several layers deep. It was a nightmare to debug and untangle.

akira2501 5y ago
What's funny to me is their solution is very natural and commonly used in unix. My guess is because signal handling on unix is much more prevalent and brings with it many of the same control flow issues you'd see in exceptions.

Matthias247 5y ago

There's nothing super special about this design. It apparently uses a single "IO reactor" which handles readiness notifications for a couple of other threads. Those might be threadpool, and/or run non-blocking code (we don't know too much about this). This design is pretty much the same that you can e.g. find in the go runtime (netpoll), the .NET core runtime, Rust multithreaded async libraries, Java nio2, etc. The only thing that was special here is that they used an APC for waking up the reactor in case new events had to be added. Its non-standard, and kind of comparable to using a signal on unix to interrupt `select` instead of a message on a preregistered socket.