7 ms·

I'm confused (but I'm also not familiar with the API). It sounds like you're saying you want to be able to read from 10k connections simultaneously while alloc

I'm confused (but I'm also not familiar with the API). It sounds like you're saying you want to be able to read from 10k connections simultaneously while allocating less than a byte of buffer space to each, so I assume I'm misunderstanding some aspect of what you're trying to say (or some aspect of the API that puts this into the correct context).

jstimpfle 8y ago
No I'm saying I want to read from N handles simultaneously using M buffers, where M <= N. Of course, the OS shouldn't write to one of my read buffers that currently contains other buffered data. But that doesn't mean that it has to be N = M, since that is a waste. I guess M = 10 * number of OS threads should usually be plenty to achieve good performance. But I'm happy to be corrected by people more experienced with systems performance.
- kbenson 8y ago
  I believe I understand. By not allocating in the api call itself you can pre-allocate your own buffers and keep track of which are in-use and not in-use yourself and use some subset of the total amount that would be allocated in each api call be reusing buffers after a particular connection is done with it. That makes sense.
zzzcpan 8y ago
On unixy systems kernel manages buffers by itself, using as little memory as possible, not allocating anything when no data is received, and userspace doesn't need to allocate any memory either until it is notified and decides to non-blockingly read the data from the kernel. Meaning that waiting for data from connections costs nothing, while on windows it requires preallocating buffer per connection. Basically the same problem that synchronous APIs have. The myth that IOCP is elegant has to die.
asveikau 8y ago
In the user mode piece, a single thread can get told "these 10K descriptors are ready for you to call read(2) upon them", then that thread can read them all using the same buffer. [In practice you might allocate more depending on what's going on, but this is a crude example.] In the IOCP model you need to allocate space for all read requests upfront, even if they're not writing you any data currently. Kernel mode would have its own buffers, so memory cost per client is not small in this example... except in terms of user-mode buffers.

jstimpfle 8y ago

No I'm saying I want to read from N handles simultaneously using M buffers, where M <= N. Of course, the OS shouldn't write to one of my read buffers that currently contains other buffered data. But that doesn't mean that it has to be N = M, since that is a waste. I guess M = 10 * number of OS threads should usually be plenty to achieve good performance. But I'm happy to be corrected by people more experienced with systems performance.

kbenson 8y ago
I believe I understand. By not allocating in the api call itself you can pre-allocate your own buffers and keep track of which are in-use and not in-use yourself and use some subset of the total amount that would be allocated in each api call be reusing buffers after a particular connection is done with it. That makes sense.

zzzcpan 8y ago

On unixy systems kernel manages buffers by itself, using as little memory as possible, not allocating anything when no data is received, and userspace doesn't need to allocate any memory either until it is notified and decides to non-blockingly read the data from the kernel. Meaning that waiting for data from connections costs nothing, while on windows it requires preallocating buffer per connection. Basically the same problem that synchronous APIs have. The myth that IOCP is elegant has to die.

asveikau 8y ago

In the user mode piece, a single thread can get told "these 10K descriptors are ready for you to call read(2) upon them", then that thread can read them all using the same buffer. [In practice you might allocate more depending on what's going on, but this is a crude example.] In the IOCP model you need to allocate space for all read requests upfront, even if they're not writing you any data currently. Kernel mode would have its own buffers, so memory cost per client is not small in this example... except in terms of user-mode buffers.