4 ms·

Would someone be kind enough to explain how a C program is structured to do asynchronous work with epoll without the use of async/await? Do you need to keep cus

Would someone be kind enough to explain how a C program is structured to do asynchronous work with epoll without the use of async/await? Do you need to keep custom structures for each network socket that maintains a snapshot of the state of how things are progressing?

j16sdiz 3y ago
> Do you need to keep custom structures for each network socket that maintains a snapshot of the state of how things are progressing? In C, yes.
willvarfar 3y ago
it was eons ago that I looked at this so I don't remember the technical details, but here's a c++ framework that used epoll https://github.com/williame/hellepoll https://github.com/williame/hellepoll
- gpderetta 3y ago
  Back in the '90s, ACE was the library that introduced/popularized reactor/proactor to C++. These days there are many libraries available. ASIO is quite popular, seastar, POCO, etc.
eqvinox 3y ago
> without the use of async/await > Do you need to keep custom structures for each network socket that maintains a snapshot of the state of how things are progressing? Yup. Everything is a callback and you dispatch those from your event loop. You're managing/implementing your own closures by (manually/explicitly) throwing all necessary data into some data structure (essentially the "async" part). Then make appropriate callback registration calls and return from your code (equivalent to "await").
latch 3y ago
I have a somewhat popular HTTP server library for Zig [1]. It started off as a thread-per-connection (with an optional thread pool), but when it became apparent that async wasn't going to be added back into the language any time soon, I switched to using epoll/kqueue. Both APIs allow you to associate arbitrary data (a `void *` in kqueue, or a union of `int/uin32_t/uint65_/void *` in epoll) with the event that you're registering. So when you're notified of the event, you can access this data. In my case, it's a big Conn struct. It contains things like the # of requests on this connection (to enforce a configured max request per connection), a timestamp where it should timeout if there's no activity. The Conn is part of an intrusive linked list, so it has a next: *Conn and prev: *Conn. But, what you're probably most curious about, is that it has a Request.State. This has a static buffer ([]u8) that can grow as needed to hold all the received data up until that point (or if we're writing the data, then the buffered data that we have to write). It's important to have a max # of connections and a max request size so you can enforce an upper limit on the maximum memory the library might use. It acts as a state machine to track up to what point it's parsed the request. (since you don't want to have to re-parse the entire request as more bytes trickle in). It's all half-baked. I can do receiving/sending asynchronously, but the application handler is called synchronously, and if that, for example, calls PG, that's probably also synchronous (since there's no async PG library in Zig). Makes me feel that any modern language needs a cohensive (as in standard library, or de facto standard) concurrency story. [1] https://github.com/karlseguin/http.zig https://github.com/karlseguin/http.zig*
- commandersaki 3y ago
  Excellent explanation and cool project. I wasn't aware that you could store a reference when you register an fd with epoll. I have used select and poll in the past, and you'd need to maintain a mapping of fd->structure somehow, and you know C doesn't come with a handy data structure like a map to make this efficient and easy when scaling to say 100k+ fds. So being able to store and retrieve a reference is incredibly useful. How do you handle the application handler code, would you run that in a separate thread to not block handling of other fds?
  - latch 3y ago
    In my project, you can start N workers, each accept(2) thanks to REUSEPORT[_LB] and manages its own epoll/kqueue. When a request is complete, the application's handler is called directly by that worker. I considered what you're suggesting: having a threadpool to dispatch application handlers on. It's obviously better. But you do have to synchronize a little more, especially if you don't trust the client. While dispatched, the client shouldn't be able to send another request, so you need to remove the READ notification for the socket and then once the response is written, re-add it. Seemed a bit tedious considering I'm hoping to throw it all out when async is re-added as a first class citizen to the language. The main benefit of my half-baked solution is that a slow or misbehaving connection won't slow (or block!) other connections. Application latency is an issue (since the worker can't processed more requests while the application handler is executing), but at least that's not open to an attack.
  - toast0 3y ago
    > and you know C doesn't come with a handy data structure like a map to make this efficient and easy when scaling to say 100k+ fds. There's plenty of map like things available; not in the language standard, but in libcs or os includes, but fds are ints, and kernel APIs that return 'new' fds are contracted to return you the least numerical fd that isn't currently in use... So you can set a max fds (the OS will tell you if you ask), and set an array of that size.
akira2501 3y ago
A convenient fact is that file descriptors are just numbers, and the lowest available number is always used when creating a new socket, so it's naturally easy to just keep all your structures in a giant array and use the fd into index into it to retrieve the structure that represents the connection. From there it's just choosing a function to dispatch to based upon epoll event and being exceptionally careful to always use non blocking IO on the underlying fd.
crabbone 3y ago
If you find Python to be easier, it has "select" module that is, on Linux, a wrapper for "epoll". Works, conceptually, in the same way.
gpderetta 3y ago
Typically explicit callbacks/continuations or state machines (not necessarily an exclusive or). Stackfull coroutines/fibers if you are feeling brave.
bregma 3y ago
> Do you need to keep custom structures for each network socket that maintains a snapshot of the state of how things are progressing? That's one way of doing it. Ideally, you track state for every connection (you need to do this even with async/await, it's just you need to be explicit) and you dispatch even handling to a thread pool using a command queue. One thread is your demultiplexor calling epoll/kqueue/MsgReceive/waitflor and all your other threads are workers in a pool, and communication is through a synchronized queue. It's not fancy. It's low overhead and fairly easy to reason about.
o11c 3y ago
Normally it's just a matter of "store a buffer of bytes received and a buffer of bytes to be sent". The "application" part doesn't do syscalls directly, just uses the buffers. Then when the application part is done (for now) you go back to the event loop which makes all the syscalls for you. There is potential for bugs if you try to be "clever" with partially parsing the buffer before all the message's data is received, but this is still far simpler than the kind of bugs we get with async/await.

j16sdiz 3y ago

> Do you need to keep custom structures for each network socket that maintains a snapshot of the state of how things are progressing? In C, yes.

willvarfar 3y ago

it was eons ago that I looked at this so I don't remember the technical details, but here's a c++ framework that used epoll https://github.com/williame/hellepoll https://github.com/williame/hellepoll

gpderetta 3y ago
Back in the '90s, ACE was the library that introduced/popularized reactor/proactor to C++. These days there are many libraries available. ASIO is quite popular, seastar, POCO, etc.

eqvinox 3y ago

> without the use of async/await > Do you need to keep custom structures for each network socket that maintains a snapshot of the state of how things are progressing? Yup. Everything is a callback and you dispatch those from your event loop. You're managing/implementing your own closures by (manually/explicitly) throwing all necessary data into some data structure (essentially the "async" part). Then make appropriate callback registration calls and return from your code (equivalent to "await").

latch 3y ago

I have a somewhat popular HTTP server library for Zig [1]. It started off as a thread-per-connection (with an optional thread pool), but when it became apparent that async wasn't going to be added back into the language any time soon, I switched to using epoll/kqueue. Both APIs allow you to associate arbitrary data (a `void *` in kqueue, or a union of `int/uin32_t/uint65_/void *` in epoll) with the event that you're registering. So when you're notified of the event, you can access this data. In my case, it's a big Conn struct. It contains things like the # of requests on this connection (to enforce a configured max request per connection), a timestamp where it should timeout if there's no activity. The Conn is part of an intrusive linked list, so it has a next: *Conn and prev: *Conn. But, what you're probably most curious about, is that it has a Request.State. This has a static buffer ([]u8) that can grow as needed to hold all the received data up until that point (or if we're writing the data, then the buffered data that we have to write). It's important to have a max # of connections and a max request size so you can enforce an upper limit on the maximum memory the library might use. It acts as a state machine to track up to what point it's parsed the request. (since you don't want to have to re-parse the entire request as more bytes trickle in). It's all half-baked. I can do receiving/sending asynchronously, but the application handler is called synchronously, and if that, for example, calls PG, that's probably also synchronous (since there's no async PG library in Zig). Makes me feel that any modern language needs a cohensive (as in standard library, or de facto standard) concurrency story. [1] https://github.com/karlseguin/http.zig https://github.com/karlseguin/http.zig*

commandersaki 3y ago
Excellent explanation and cool project. I wasn't aware that you could store a reference when you register an fd with epoll. I have used select and poll in the past, and you'd need to maintain a mapping of fd->structure somehow, and you know C doesn't come with a handy data structure like a map to make this efficient and easy when scaling to say 100k+ fds. So being able to store and retrieve a reference is incredibly useful. How do you handle the application handler code, would you run that in a separate thread to not block handling of other fds?
- latch 3y ago
  In my project, you can start N workers, each accept(2) thanks to REUSEPORT[_LB] and manages its own epoll/kqueue. When a request is complete, the application's handler is called directly by that worker. I considered what you're suggesting: having a threadpool to dispatch application handlers on. It's obviously better. But you do have to synchronize a little more, especially if you don't trust the client. While dispatched, the client shouldn't be able to send another request, so you need to remove the READ notification for the socket and then once the response is written, re-add it. Seemed a bit tedious considering I'm hoping to throw it all out when async is re-added as a first class citizen to the language. The main benefit of my half-baked solution is that a slow or misbehaving connection won't slow (or block!) other connections. Application latency is an issue (since the worker can't processed more requests while the application handler is executing), but at least that's not open to an attack.
- toast0 3y ago
  > and you know C doesn't come with a handy data structure like a map to make this efficient and easy when scaling to say 100k+ fds. There's plenty of map like things available; not in the language standard, but in libcs or os includes, but fds are ints, and kernel APIs that return 'new' fds are contracted to return you the least numerical fd that isn't currently in use... So you can set a max fds (the OS will tell you if you ask), and set an array of that size.

akira2501 3y ago

A convenient fact is that file descriptors are just numbers, and the lowest available number is always used when creating a new socket, so it's naturally easy to just keep all your structures in a giant array and use the fd into index into it to retrieve the structure that represents the connection. From there it's just choosing a function to dispatch to based upon epoll event and being exceptionally careful to always use non blocking IO on the underlying fd.

crabbone 3y ago

If you find Python to be easier, it has "select" module that is, on Linux, a wrapper for "epoll". Works, conceptually, in the same way.

gpderetta 3y ago

Typically explicit callbacks/continuations or state machines (not necessarily an exclusive or). Stackfull coroutines/fibers if you are feeling brave.

bregma 3y ago

> Do you need to keep custom structures for each network socket that maintains a snapshot of the state of how things are progressing? That's one way of doing it. Ideally, you track state for every connection (you need to do this even with async/await, it's just you need to be explicit) and you dispatch even handling to a thread pool using a command queue. One thread is your demultiplexor calling epoll/kqueue/MsgReceive/waitflor and all your other threads are workers in a pool, and communication is through a synchronized queue. It's not fancy. It's low overhead and fairly easy to reason about.

o11c 3y ago

Normally it's just a matter of "store a buffer of bytes received and a buffer of bytes to be sent". The "application" part doesn't do syscalls directly, just uses the buffers. Then when the application part is done (for now) you go back to the event loop which makes all the syscalls for you. There is potential for bugs if you try to be "clever" with partially parsing the buffer before all the message's data is received, but this is still far simpler than the kind of bugs we get with async/await.