5 ms·

If you get stuck in read(), you can't do neat things like waking up when it's time to kick a client for being idle, doing other housekeeping, or cleanly shuttin

If you get stuck in read(), you can't do neat things like waking up when it's time to kick a client for being idle, doing other housekeeping, or cleanly shutting down the whole thing in a timely fashion. When I ^C the server, it sends the same wake condvar-poke but it twiddles the flags so the worker shuts down instead.

pdonis 6y ago
Yes, using epoll with nonblocking I/O is better than blocking I/O on each worker thread. But that basically means that you are doing asynchronous programming--i.e., the exact same thing that the wonky Python/Gunicorn stack you described is doing! You're just doing it with better attention to important details. Here, to me, is the key item: The "listener" thread owns all of the file descriptors (listeners and clients both), and manages a single epoll set to watch over them. This is exactly what any async server does: it centralizes all the file descriptor management and handling in one place, and only uses workers (whether they are threads or "green threads" or whatever) to read from/write to fd's that are marked as ready in the epoll set. For your case, unless I'm misreading something, what the workers are doing in between the read/write is CPU intensive (or at least it's CPU work and not I/O work, even though it's not very "intensive" CPU work), so actual OS threads are a better choice for the workers since you can't rely on cooperative scheduling. If what the workers were doing was I/O work (for example, sending a request to a remote database and waiting for a response), "green threads" would work fine (since their only real purpose would be to organize the I/O--the actual fd's are going to be managed by the central server that manages all the fd's and checks which ones are ready for read/write). And one definitely should not try to run "green threads" for the same server in multiple O/S threads (or worse still, multiple OS processes). For an I/O bound server, one shouldn't need to anyway.
- underdeserver 6y ago
  Better attention to important details is what makes or breaks a library.
  - swsieber 6y ago
    Exhibit A: Dropbox (attention to detail and executing it)
the8472 6y ago
> If you get stuck in read(), you can't do neat things like waking up when it's time to kick a client for being idle Totally possible with another thread acting as watchdog timer and sending a signal which causes the read to return with EINTR which can then check a flag whether it should retry or abort. And that's for file IO. For socket IO you can just set it to non-blocking.
- kragen 6y ago
  File I/O is usually not interruptible with signals. An alternative to putting the watchdog timer in another thread is to use alarm(2) and use the kernel's built-in watchdog timer, and the default behavior for SIGALRM is probably adequate. This might be easier than non-blocking I/O.
simfoo 6y ago
Good point. Would still be possible with the threads being blocked in a read() but that would require signals and more logic in the threads, so a central multiplexing and coordinating thread seems like a cleaner solution.

pdonis 6y ago

Yes, using epoll with nonblocking I/O is better than blocking I/O on each worker thread. But that basically means that you are doing asynchronous programming--i.e., the exact same thing that the wonky Python/Gunicorn stack you described is doing! You're just doing it with better attention to important details. Here, to me, is the key item: The "listener" thread owns all of the file descriptors (listeners and clients both), and manages a single epoll set to watch over them. This is exactly what any async server does: it centralizes all the file descriptor management and handling in one place, and only uses workers (whether they are threads or "green threads" or whatever) to read from/write to fd's that are marked as ready in the epoll set. For your case, unless I'm misreading something, what the workers are doing in between the read/write is CPU intensive (or at least it's CPU work and not I/O work, even though it's not very "intensive" CPU work), so actual OS threads are a better choice for the workers since you can't rely on cooperative scheduling. If what the workers were doing was I/O work (for example, sending a request to a remote database and waiting for a response), "green threads" would work fine (since their only real purpose would be to organize the I/O--the actual fd's are going to be managed by the central server that manages all the fd's and checks which ones are ready for read/write). And one definitely should not try to run "green threads" for the same server in multiple O/S threads (or worse still, multiple OS processes). For an I/O bound server, one shouldn't need to anyway.

underdeserver 6y ago
Better attention to important details is what makes or breaks a library.
- swsieber 6y ago
  Exhibit A: Dropbox (attention to detail and executing it)

the8472 6y ago

> If you get stuck in read(), you can't do neat things like waking up when it's time to kick a client for being idle Totally possible with another thread acting as watchdog timer and sending a signal which causes the read to return with EINTR which can then check a flag whether it should retry or abort. And that's for file IO. For socket IO you can just set it to non-blocking.

kragen 6y ago
File I/O is usually not interruptible with signals. An alternative to putting the watchdog timer in another thread is to use alarm(2) and use the kernel's built-in watchdog timer, and the default behavior for SIGALRM is probably adequate. This might be easier than non-blocking I/O.

simfoo 6y ago

Good point. Would still be possible with the threads being blocked in a read() but that would require signals and more logic in the threads, so a central multiplexing and coordinating thread seems like a cleaner solution.