3 ms·

Obviously you will write faster code if you use directly OS functions, but your code will be less portable.

by octopus 14y ago

exDM69 14y ago
That is very unfortunate, but sometimes performance goes over portability. I think it was a major omission to leave out async I/O abstractions from C11 and C++11 standard libraries. But you can't do much with only the standard library anyway. In addition to async filesystem i/o, it's completely lacking network sockets (async or otherwise) and other important, almost standard functionality. And besides, aio is a POSIX standard, so it should be fairly widely supported. With POSIX aio + Win32 async i/o you should get quite good portability. And in any case, it's just a bad idea to have a ton of threads waiting for synchronous I/O completion. If using async i/o is out of the question, use a thread pool to keep your resource consumption sane.
- JoeAltmaier 14y ago
  Thread pools require tuning. That means you can get it wrong, or what was right on one platform is later wrong on another. In any case, you have to burn human resources making it right. Again, why is it a bad idea to use threads? Your OS doesn't feel any pain, its quite willing to support 100's of threads. While async i/o is cool, if the i/o context of each thread is quite different then again you're 'manually' handling context when fielding async, which can be worse in terms of hardware caches etc. associated with the thread that fields the async call.
  - Nursie 14y ago
    It's not bad to use threads. Historically it has been bad to use a heck of a lot of threads because you add a lot of context switching overhead that you could otherwise avoid, as the OS tries to schedule everything to get some time and thus spends some cycles suspending and restoring active threads. If you have many hundreds of threads then this will involve fetching thread contexts from main memory (it may even with a few, I'm not sure on this). So the theory is that if don't absolutely need all of these things to run concurrently then you can design your workload to use these cycles instead of wasting them on switching back and forth, and get your work done faster. I'm sure that hardware and operating systems are designed with this in mind and the effect is reduced, but it's still going to be a factor. And I know we all have machines with gigabytes and gigabytes of RAM these days, but limiting the number of threads used is going to have a huge effect on memory footprint, which is still important in a lot of places (embedded/mobile etc). Thread pools can/could be written to auto-tune, depending on the nature of the workload. The human is only comparing times over multiple runs, you can do this programmatically though this is more coding overhead. It does need to tune to the specific machine though, as all sorts of factors will affect the result. I'm not going to pretend they're a panacea but they can still be useful as they provide flexibility and a way around the context-switch costs. EDIT - Also thread pools don't require tuning, and are not even necessary at all in this situation, people (self included) are just reacting to the idea of throwing orders of magnitude more threads at a system than it has processors, and expecting optimal results.
    - JoeAltmaier 14y ago
      Thread switch time does not scale to the # of threads involved. Switching between 100 threads each blocked on i/o is exactly as efficient time-wise as switching between two threads ping-ponging the same i/o. Memory footprint is somewhat applicable - often each has a (virtual) megabyte stack. So in 32-bit address mode you could run out of virtual memory!
      - exDM69 14y ago
        > Thread switch time does not scale to the # of threads involved. True, but the number of context switches does increase and that in turn decreases the ratio of useful work vs. context switch overhead. Also the secondary costs of context switching, like flushing caches increase. If there are hundreds of threads, the chances that a thread is woken up with warm caches decreases.
        JoeAltmaier 14y ago
        Again, my point Was that context switching is identical. That is equal to the # io completions and nothing else.

Obviously you will write faster code if you use directly OS functions, but your code will be less portable.

by octopus 14y ago

Obviously you will write faster code if you use directly OS functions, but your code will be less portable.

That is very unfortunate, but sometimes performance goes over portability. I think it was a major omission to leave out async I/O abstractions from C11 and C++11 standard libraries. But you can't do much with only the standard library anyway. In addition to async filesystem i/o, it's completely lacking network sockets (async or otherwise) and other important, almost standard functionality. And besides, aio is a POSIX standard, so it should be fairly widely supported. With POSIX aio + Win32 async i/o you should get quite good portability. And in any case, it's just a bad idea to have a ton of threads waiting for synchronous I/O completion. If using async i/o is out of the question, use a thread pool to keep your resource consumption sane.

JoeAltmaier 14y ago
Thread pools require tuning. That means you can get it wrong, or what was right on one platform is later wrong on another. In any case, you have to burn human resources making it right. Again, why is it a bad idea to use threads? Your OS doesn't feel any pain, its quite willing to support 100's of threads. While async i/o is cool, if the i/o context of each thread is quite different then again you're 'manually' handling context when fielding async, which can be worse in terms of hardware caches etc. associated with the thread that fields the async call.
- Nursie 14y ago
  It's not bad to use threads. Historically it has been bad to use a heck of a lot of threads because you add a lot of context switching overhead that you could otherwise avoid, as the OS tries to schedule everything to get some time and thus spends some cycles suspending and restoring active threads. If you have many hundreds of threads then this will involve fetching thread contexts from main memory (it may even with a few, I'm not sure on this). So the theory is that if don't absolutely need all of these things to run concurrently then you can design your workload to use these cycles instead of wasting them on switching back and forth, and get your work done faster. I'm sure that hardware and operating systems are designed with this in mind and the effect is reduced, but it's still going to be a factor. And I know we all have machines with gigabytes and gigabytes of RAM these days, but limiting the number of threads used is going to have a huge effect on memory footprint, which is still important in a lot of places (embedded/mobile etc). Thread pools can/could be written to auto-tune, depending on the nature of the workload. The human is only comparing times over multiple runs, you can do this programmatically though this is more coding overhead. It does need to tune to the specific machine though, as all sorts of factors will affect the result. I'm not going to pretend they're a panacea but they can still be useful as they provide flexibility and a way around the context-switch costs. EDIT - Also thread pools don't require tuning, and are not even necessary at all in this situation, people (self included) are just reacting to the idea of throwing orders of magnitude more threads at a system than it has processors, and expecting optimal results.
  - JoeAltmaier 14y ago
    Thread switch time does not scale to the # of threads involved. Switching between 100 threads each blocked on i/o is exactly as efficient time-wise as switching between two threads ping-ponging the same i/o. Memory footprint is somewhat applicable - often each has a (virtual) megabyte stack. So in 32-bit address mode you could run out of virtual memory!
    - exDM69 14y ago
      > Thread switch time does not scale to the # of threads involved. True, but the number of context switches does increase and that in turn decreases the ratio of useful work vs. context switch overhead. Also the secondary costs of context switching, like flushing caches increase. If there are hundreds of threads, the chances that a thread is woken up with warm caches decreases.
      - JoeAltmaier 14y ago
        Again, my point Was that context switching is identical. That is equal to the # io completions and nothing else.