5 ms·

By blocking they mean that it can take ballpark non-volatile storage times instead of ballpark RAM times

xoranth 2y ago
I believe they mean that since it bypasses the (Tokio) scheduler, so if you use it in async code you lose the main benefit of async code (namely, the scheduler is able to switch to some other task while waiting for IO to complete.). Basically the same behavior you'd get if you called a blocking syscall directly.
guerrilla 2y ago
GP is aware. mmap makes files act like memory. Memory is always synchronous, thus blocking, so mmaped files are always blocking. I'm surprised OP even found this surprising. It should be completely obvious.
- dataflow 2y ago
  At first I thought the title meant the mmap() call itself blocks, which I figured could be slightly surprising. But it seems they're referring to I/O on the mapped file? I'm also baffled, how could it possibly not block?
  - usefulcat 2y ago
    > how could it possibly not block? When the bytes being read are already in the cache. Hence the later part of the article where the author shows that reading mapped memory can be significantly faster.
    - magicalhippo 2y ago
      It still blocks. It just completes orders of magnitudes faster.
      - icedchai 2y ago
        Do you consider reading from a normal array (one not backed by a memory mapped file) to also be blocking?
        okr 2y ago
        Ha. Exactly.
        magicalhippo 2y ago
        In the languages and platforms I use, absolutely yes. Do you have some examples where a normal memory read is async?
        icedchai 2y ago
        Your definition of blocking is a bit different from my own. Synchronous is not always blocking. If the data is there, ready to go, there is no "blocking." If you consider all memory reads to be "blocking", then everything must be "blocking". The executable code must, after all, be read by the processor. In an extreme case, the entire executable could be paged out to disk! This interpretation is not what most people mean by "blocking."
        magicalhippo 2y ago
        Fair point. I guess I conflate the two, because what's interesting to me, most of the time, is where does the control flow switch. I never rely on synchronous IO being non-blocking when writing regular code (ie not embedded). As such reading from cache (non-blocking) vs disk (blocking) doesn't matter that much, as such. It's synchronous and that's all I need to reason about how it behaves. If I need it to be non-blocking, ie playing audio from a file, then I need to ensure it via other means (pre-loading buffer in a background thread, etc etc). edit: And if I really need it not to block, the buffer needs to reside in the non-paged pool. Otherwise it can get swapped to disk.
        juped 2y ago
        you don't yield to your cooperative-multitasking runtime during reading from it, which is obviously what everyone in this thread means, and it's not helpful to start telling them "you're using the word blocking wrong" apropos of nothing
  - icedchai 2y ago
    One trick is to read the file into memory at application startup. All data is paged in, so it's hot and ready to go: no page faults. In the early 2000's, I worked on a near real time system that used memory mapped I/O. At app startup, several gigabytes were read into memory. It never blocked under normal circumstances (in production) since the systems were provisioned with enough memory.
    - dataflow 2y ago
      But that requires knowing your RAM is big enough to fit the file. It can't work in general.
  - anonymoushn 2y ago
    Well, the OP could probably get their benchmarks to run faster if they passed MAP_POPULATE, which would make the mmap call block for longer. In a pedantic sense, the mmap call is already blocking, because any system call takes longer than e.g. stuffing an sqe onto a queue and then making 0 syscalls, and it could take a variable amount of time depending on factors beyond that one process's control. I don't think anyone actually needs to offload their non-MAP_POPULATE mmaps to a separate thread or whatever though.
  - dbaupp 2y ago
    Yes, it couldn't not block, it's obvious... but I've encountered non-trivial amounts of "magical thinking" around both async/await ("go-fast juice") and mmap ("go-fast juice") separately, so the intersection surely has a bunch of magical thinking too, where people haven't taken the time to properly think through what's going on. Hence, my investigation to try to make the "obvious" conclusion obvious to more people. (Author here)
- jerf 2y ago
  The term "blocking" has diverged between various communities and it is important to recognize those differences or you'll have dozens of people talking past each other for hundreds of messages as they all say "blocking" and think they mean the same thing, and then get very confused and angry at all the other people who are so obviously wrong (and in their context, they are) but just can't see it. It is obvious that a given "execution context", which is my generalized term for a thread and an async job and anything else of a similar nature, when it reaches for a value from an mmap'd file will be blocked until it is available. However, different communities have different ideas of an "execution context". Threaded language users tend to see them as threads, so while a given thread may be blocked the rest of the program can generally proceed. (Although historically the full story around file operations and what other threads can proceed past has been quite complicated.) Async users on the other hand are surprised here because the operation is blocking their entire executor, even though in principle it ought to be able to proceed with some other context. Because it's invisible to the executor, it isn't able to context-switch. In this case, the threaded world view is reasonably "obvious" but it can be non-obvious that a given async environment may not be able to task switch and it may freeze an entire executor, and since "one executor" is still a fairly common scenario, the entire OS process. (I am expressing no opinion about whether it must block an executor. System calls come with a lot of flags nowadays and for all I know there's some way an async executor could "catch" a mapped access and have an opportunity to switch. I am taking the original article's implicit claim that there isn't one happening in their particular environment at face value.) As long as you do not distinguish how various communities use the term "blocking", you will get very, very deeply nested threads full of arguments about something that, if you just are careful with your terminology, isn't complicated for anyone from any subculture to understand.
  - deleted 2y ago
    [deleted]
- dbaupp 2y ago
  Yeah, of course a synchronous call that might block the thread is blocking, I agree... but, if I didn't have the context of "we're in a comment thread about a blog post about mmap", I'm pretty sure I wouldn't flag `x[i]` on `x: &[u8]` (or any other access) as a synchronous call that is worth worrying about. Hence the discussion of subtlety in https://huonw.github.io/blog/2024/08/async-hazard-mmap/#at-a-distance https://huonw.github.io/blog/2024/08/async-hazard-mmap/#at-a... It's obvious when pointed out, but I don't think it's obvious without the context of "there's an mmap nearby". (Author here, this behaviour wasn't surprising to me, but felt subtle enough to be worth investigating, and get some concrete data.)
bhawks 2y ago
Same thing would happen if your memory has been paged to disk too.

I believe they mean that since it bypasses the (Tokio) scheduler, so if you use it in async code you lose the main benefit of async code (namely, the scheduler is able to switch to some other task while waiting for IO to complete.). Basically the same behavior you'd get if you called a blocking syscall directly.

guerrilla 2y ago

GP is aware. mmap makes files act like memory. Memory is always synchronous, thus blocking, so mmaped files are always blocking. I'm surprised OP even found this surprising. It should be completely obvious.

dataflow 2y ago
At first I thought the title meant the mmap() call itself blocks, which I figured could be slightly surprising. But it seems they're referring to I/O on the mapped file? I'm also baffled, how could it possibly not block?
- usefulcat 2y ago
  > how could it possibly not block? When the bytes being read are already in the cache. Hence the later part of the article where the author shows that reading mapped memory can be significantly faster.
  - magicalhippo 2y ago
    It still blocks. It just completes orders of magnitudes faster.
    - icedchai 2y ago
      Do you consider reading from a normal array (one not backed by a memory mapped file) to also be blocking?
      - okr 2y ago
        Ha. Exactly.
      - magicalhippo 2y ago
        In the languages and platforms I use, absolutely yes. Do you have some examples where a normal memory read is async?
        icedchai 2y ago
        Your definition of blocking is a bit different from my own. Synchronous is not always blocking. If the data is there, ready to go, there is no "blocking." If you consider all memory reads to be "blocking", then everything must be "blocking". The executable code must, after all, be read by the processor. In an extreme case, the entire executable could be paged out to disk! This interpretation is not what most people mean by "blocking."
        magicalhippo 2y ago
        Fair point. I guess I conflate the two, because what's interesting to me, most of the time, is where does the control flow switch. I never rely on synchronous IO being non-blocking when writing regular code (ie not embedded). As such reading from cache (non-blocking) vs disk (blocking) doesn't matter that much, as such. It's synchronous and that's all I need to reason about how it behaves. If I need it to be non-blocking, ie playing audio from a file, then I need to ensure it via other means (pre-loading buffer in a background thread, etc etc). edit: And if I really need it not to block, the buffer needs to reside in the non-paged pool. Otherwise it can get swapped to disk.
        juped 2y ago
        you don't yield to your cooperative-multitasking runtime during reading from it, which is obviously what everyone in this thread means, and it's not helpful to start telling them "you're using the word blocking wrong" apropos of nothing
- icedchai 2y ago
  One trick is to read the file into memory at application startup. All data is paged in, so it's hot and ready to go: no page faults. In the early 2000's, I worked on a near real time system that used memory mapped I/O. At app startup, several gigabytes were read into memory. It never blocked under normal circumstances (in production) since the systems were provisioned with enough memory.
  - dataflow 2y ago
    But that requires knowing your RAM is big enough to fit the file. It can't work in general.
- anonymoushn 2y ago
  Well, the OP could probably get their benchmarks to run faster if they passed MAP_POPULATE, which would make the mmap call block for longer. In a pedantic sense, the mmap call is already blocking, because any system call takes longer than e.g. stuffing an sqe onto a queue and then making 0 syscalls, and it could take a variable amount of time depending on factors beyond that one process's control. I don't think anyone actually needs to offload their non-MAP_POPULATE mmaps to a separate thread or whatever though.
- dbaupp 2y ago
  Yes, it couldn't not block, it's obvious... but I've encountered non-trivial amounts of "magical thinking" around both async/await ("go-fast juice") and mmap ("go-fast juice") separately, so the intersection surely has a bunch of magical thinking too, where people haven't taken the time to properly think through what's going on. Hence, my investigation to try to make the "obvious" conclusion obvious to more people. (Author here)
jerf 2y ago
The term "blocking" has diverged between various communities and it is important to recognize those differences or you'll have dozens of people talking past each other for hundreds of messages as they all say "blocking" and think they mean the same thing, and then get very confused and angry at all the other people who are so obviously wrong (and in their context, they are) but just can't see it. It is obvious that a given "execution context", which is my generalized term for a thread and an async job and anything else of a similar nature, when it reaches for a value from an mmap'd file will be blocked until it is available. However, different communities have different ideas of an "execution context". Threaded language users tend to see them as threads, so while a given thread may be blocked the rest of the program can generally proceed. (Although historically the full story around file operations and what other threads can proceed past has been quite complicated.) Async users on the other hand are surprised here because the operation is blocking their entire executor, even though in principle it ought to be able to proceed with some other context. Because it's invisible to the executor, it isn't able to context-switch. In this case, the threaded world view is reasonably "obvious" but it can be non-obvious that a given async environment may not be able to task switch and it may freeze an entire executor, and since "one executor" is still a fairly common scenario, the entire OS process. (I am expressing no opinion about whether it must block an executor. System calls come with a lot of flags nowadays and for all I know there's some way an async executor could "catch" a mapped access and have an opportunity to switch. I am taking the original article's implicit claim that there isn't one happening in their particular environment at face value.) As long as you do not distinguish how various communities use the term "blocking", you will get very, very deeply nested threads full of arguments about something that, if you just are careful with your terminology, isn't complicated for anyone from any subculture to understand.
- deleted 2y ago
  [deleted]
dbaupp 2y ago
Yeah, of course a synchronous call that might block the thread is blocking, I agree... but, if I didn't have the context of "we're in a comment thread about a blog post about mmap", I'm pretty sure I wouldn't flag `x[i]` on `x: &[u8]` (or any other access) as a synchronous call that is worth worrying about. Hence the discussion of subtlety in https://huonw.github.io/blog/2024/08/async-hazard-mmap/#at-a-distance https://huonw.github.io/blog/2024/08/async-hazard-mmap/#at-a... It's obvious when pointed out, but I don't think it's obvious without the context of "there's an mmap nearby". (Author here, this behaviour wasn't surprising to me, but felt subtle enough to be worth investigating, and get some concrete data.)

bhawks 2y ago

Same thing would happen if your memory has been paged to disk too.