3 ms·

I haven’t developed much for Linux, but I think it’s a good idea to replace your std::vector<ctx::continuation> fibers with std::unordered_map<int, ctx::continu

I haven’t developed much for Linux, but I think it’s a good idea to replace your std::vector<ctx::continuation> fibers with std::unordered_map<int, ctx::continuation> fibers. Here’s why: https://stackoverflow.com/a/9376493/126995 https://stackoverflow.com/a/9376493/126995

drfuchs 9y ago
That StackOverflow comment is wrong. Linux guarantees that for open(2) / creat(2) / socket(2) / etc. "[t]he file descriptor returned by a successful call will be the lowest-numbered file descriptor not currently open for the process." This has been the case for the entire 40+ year history of Unix, and was certainly relied upon in the days when select(2) could only handle file descriptor values that were less than 32. See the man page at, e.g., http://man7.org/linux/man-pages/man2/socket.2.html http://man7.org/linux/man-pages/man2/socket.2.html from which the quote above is taken. Note that I'm not expressing any opinion on whether to use a vector or an unordered_map in this particular application.
- Const-me 9y ago
  > Linux guarantees that for open(2) / creat(2) / socket(2) / etc. Not “etc.”, accept(2) that creates the sockets in that code doesn’t guarantee that. See the man page at, e.g., http://man7.org/linux/man-pages/man2/accept.2.html http://man7.org/linux/man-pages/man2/accept.2.html
  - drfuchs 9y ago
    Hmmm. Looking further, the dup(2) man page also mentions "lowest," but pipe(2) doesn't. A documentation oversight, I claim; after all, what would be the point of the kernel not using a single function for "get me an available fd" everywhere that it needs one? Not as reassuring as finding it in the specs, though; here's what http://pubs.opengroup.org/onlinepubs/9699919799/ http://pubs.opengroup.org/onlinepubs/9699919799/ has to say: "2.14. File Descriptor Allocation All functions that open one or more file descriptors shall, unless specified otherwise, atomically allocate the lowest numbered available (that is, not already open in the calling process) file descriptor at the time of each allocation. Where a single function allocates two file descriptors (for example, pipe() or socketpair()), the allocations may be independent and therefore applications should not expect them to have adjacent values or depend on which has the higher value."
    - gravypod 9y ago
      If your application is long running and lives under a snapshot and restore system you may find this behavior. Just a guess.
    - Const-me 9y ago
      > what would be the point of the kernel not using a single function for "get me an available fd" everywhere that it needs one? Multithreading. To return lowest-numbered file descriptor, calls to that single function need to be serialized. On multi-core, and especially on NUMA, I can see how that might hit the performance. Apps don’t normally create thousands of files, or pipes, or client sockets per second. However, for some server software, accepting thousands of sockets per second is normal. Update, see also multi-queue NICs: https://www.kernel.org/doc/Documentation/networking/scaling.txt https://www.kernel.org/doc/Documentation/networking/scaling....
noselasd 9y ago
Note that the stackoverflow post does not technically apply. accept() finds the smalles available file descriptor - it will never suddenly hand you a file descriptor of a huge integer value. You can also control this with the "max open file" ulimit setting. This is by default often 1024 - meaning your process can have max 1024 open file descriptors, and the max value of a descriptor will be 1023. Even if you need to support one or two orders of magnitude more file descriptors, a vector would likely be more efficient due to better memory locality.
- Const-me 9y ago
  > a vector would likely be more efficient due to better memory locality Vector’s memory locality only matters when you’re iterating over the elements. For this case it’s essentially a random index. Besides, unordered_map memory locality is fixable: https://github.com/Const-me/CollectionMicrobench https://github.com/Const-me/CollectionMicrobench
  - matt42 9y ago
    I'll try and swith to unordered_map if it has a negligible impact on performances.

drfuchs 9y ago

That StackOverflow comment is wrong. Linux guarantees that for open(2) / creat(2) / socket(2) / etc. "[t]he file descriptor returned by a successful call will be the lowest-numbered file descriptor not currently open for the process." This has been the case for the entire 40+ year history of Unix, and was certainly relied upon in the days when select(2) could only handle file descriptor values that were less than 32. See the man page at, e.g., http://man7.org/linux/man-pages/man2/socket.2.html http://man7.org/linux/man-pages/man2/socket.2.html from which the quote above is taken. Note that I'm not expressing any opinion on whether to use a vector or an unordered_map in this particular application.

Const-me 9y ago
> Linux guarantees that for open(2) / creat(2) / socket(2) / etc. Not “etc.”, accept(2) that creates the sockets in that code doesn’t guarantee that. See the man page at, e.g., http://man7.org/linux/man-pages/man2/accept.2.html http://man7.org/linux/man-pages/man2/accept.2.html
- drfuchs 9y ago
  Hmmm. Looking further, the dup(2) man page also mentions "lowest," but pipe(2) doesn't. A documentation oversight, I claim; after all, what would be the point of the kernel not using a single function for "get me an available fd" everywhere that it needs one? Not as reassuring as finding it in the specs, though; here's what http://pubs.opengroup.org/onlinepubs/9699919799/ http://pubs.opengroup.org/onlinepubs/9699919799/ has to say: "2.14. File Descriptor Allocation All functions that open one or more file descriptors shall, unless specified otherwise, atomically allocate the lowest numbered available (that is, not already open in the calling process) file descriptor at the time of each allocation. Where a single function allocates two file descriptors (for example, pipe() or socketpair()), the allocations may be independent and therefore applications should not expect them to have adjacent values or depend on which has the higher value."
  - gravypod 9y ago
    If your application is long running and lives under a snapshot and restore system you may find this behavior. Just a guess.
  - Const-me 9y ago
    > what would be the point of the kernel not using a single function for "get me an available fd" everywhere that it needs one? Multithreading. To return lowest-numbered file descriptor, calls to that single function need to be serialized. On multi-core, and especially on NUMA, I can see how that might hit the performance. Apps don’t normally create thousands of files, or pipes, or client sockets per second. However, for some server software, accepting thousands of sockets per second is normal. Update, see also multi-queue NICs: https://www.kernel.org/doc/Documentation/networking/scaling.txt https://www.kernel.org/doc/Documentation/networking/scaling....

noselasd 9y ago

Note that the stackoverflow post does not technically apply. accept() finds the smalles available file descriptor - it will never suddenly hand you a file descriptor of a huge integer value. You can also control this with the "max open file" ulimit setting. This is by default often 1024 - meaning your process can have max 1024 open file descriptors, and the max value of a descriptor will be 1023. Even if you need to support one or two orders of magnitude more file descriptors, a vector would likely be more efficient due to better memory locality.

Const-me 9y ago
> a vector would likely be more efficient due to better memory locality Vector’s memory locality only matters when you’re iterating over the elements. For this case it’s essentially a random index. Besides, unordered_map memory locality is fixable: https://github.com/Const-me/CollectionMicrobench https://github.com/Const-me/CollectionMicrobench
- matt42 9y ago
  I'll try and swith to unordered_map if it has a negligible impact on performances.