2 ms·

You can do that with a MAP_ANONYMOUS | MAP_SHARED mapping too: that kind of mapping is writable by both parent and child, but isn't backed by a disk file and so

You can do that with a MAP_ANONYMOUS | MAP_SHARED mapping too: that kind of mapping is writable by both parent and child, but isn't backed by a disk file and so can't be truncated or surprise-removed. The article's points about mmap infelicity applies mostly to mappings of disk files. Anonymous mappings don't have the same problems.

tus87 8y ago
Yes, but if the file is on local disk its exactly the same situation as the page file. OP seems to be trying to cope with NFS errors....but if you are using mmap on NFS you have bigger problems...
0db532a0 8y ago
The article points out the infelicity as applied to mappings of files into multi-threaded programmes. What fpoling suggests is a solution to the POSIX signals problem where anonymous mmaps are not an option.
_wmd 8y ago
Anonymous mappings are backed by swap and may be overcommitted, it's still possible to catch signals in a wide variety of circumstances There is probably enough evidence in this thread to use it as a reference for why typical apps should avoid mmap whenever possible -- it's clear almost nobody fully understands it
- gpderetta 8y ago
  > Anonymous mappings are backed by swap and may be overcommitted, So is normal memory. Many allocators today even use mmap internally.
  - deleted 8y ago
    [deleted]
  - quotemstr 8y ago
    All* allocation is mmap, really. All mmap does is dedicate a region of address space to some kind of backing storage. The particular kind of backing storage makes all the difference. The problem is that people colloquially use "mmap" to mean "mmap of a conventional disk file" and don't mean all the other kinds of mmap out there, so discussions can become confusing. * Ther's sbrk too, but it's just a fancy legacy path that amounts to the same thing as anonymous mmap
- klodolph 8y ago
  > Anonymous mappings are backed by swap and may be overcommitted, it's still possible to catch signals in a wide variety of circumstances Anonymous mappings won't cause signals, they'll trigger the OOM killer. Remember that malloc() is just a fancy wrapper for mmap() (and sbrk()).
  - _wmd 8y ago
    If a process was swapped out and a fault fails to bring a page back due to an IO error, you can at least catch (I think) SIGBUS. But this just reinforces the point: nobody really understands virtual memory, even people like us that think they do
    - klodolph 8y ago
      Okay, but I'd say the correct thing to do is let SIGBUS kill the process. You can at least expect to handle SIGBUS for a file you've mapped.
    - tntn 8y ago
      So should we extend your conclusion above to the following? "There is probably enough evidence in this thread to use it as a reference for why typical apps should avoid virtual memory whenever possible -- it's clear almost nobody fully understands it" I'd suggest that is ludicrous, and for the same reason your original conclusion is also excessive.
      - _wmd 8y ago
        It is not constructive to form a sweeping generalization from a statement and then claim the sweeping generalization is ludicrous, implying the original statement is ludicrous. :) My first comment was in reply to one claiming anonymous memory did not have the same problems as file-backed memory, indicating the parent did not understand they are the same thing. The subsequent reply was to another comment continuing to claim anonymous memory was somehow safer, both instances supporting the notion that most people in this thread don't seem to understand mmap at all. What we're examining is a powerful (and consequently hazardous) OS feature that often provides only marginal performance improvement, yet introduces many exotic error paths into a program that have their own exotic problems (memory access in thread A can raise SEGV in thread B, async-signal safety), that 7 hours' commenting has not been sufficient to fully capture. This thread is full of upvoted miscomprehension, bad advice (spawn a child to deal with SEGV!?), obviously incorrect solutions (signalfd), and yet still manages to completely omit some critical characteristics of mmap, for example, that faults take a VM-global semaphore -- mmap can easily destroy multithreaded app performance in a way read() is immune to, because nobody expects file IO in one thread to cause malloc() latency in another. If this isn't evidence for "avoid this feature wherever possible", I really don't know what is.

tus87 8y ago

Yes, but if the file is on local disk its exactly the same situation as the page file. OP seems to be trying to cope with NFS errors....but if you are using mmap on NFS you have bigger problems...

0db532a0 8y ago

The article points out the infelicity as applied to mappings of files into multi-threaded programmes. What fpoling suggests is a solution to the POSIX signals problem where anonymous mmaps are not an option.

_wmd 8y ago

Anonymous mappings are backed by swap and may be overcommitted, it's still possible to catch signals in a wide variety of circumstances There is probably enough evidence in this thread to use it as a reference for why typical apps should avoid mmap whenever possible -- it's clear almost nobody fully understands it

gpderetta 8y ago
> Anonymous mappings are backed by swap and may be overcommitted, So is normal memory. Many allocators today even use mmap internally.
- deleted 8y ago
  [deleted]
- quotemstr 8y ago
  All* allocation is mmap, really. All mmap does is dedicate a region of address space to some kind of backing storage. The particular kind of backing storage makes all the difference. The problem is that people colloquially use "mmap" to mean "mmap of a conventional disk file" and don't mean all the other kinds of mmap out there, so discussions can become confusing. * Ther's sbrk too, but it's just a fancy legacy path that amounts to the same thing as anonymous mmap
klodolph 8y ago
> Anonymous mappings are backed by swap and may be overcommitted, it's still possible to catch signals in a wide variety of circumstances Anonymous mappings won't cause signals, they'll trigger the OOM killer. Remember that malloc() is just a fancy wrapper for mmap() (and sbrk()).
- _wmd 8y ago
  If a process was swapped out and a fault fails to bring a page back due to an IO error, you can at least catch (I think) SIGBUS. But this just reinforces the point: nobody really understands virtual memory, even people like us that think they do
  - klodolph 8y ago
    Okay, but I'd say the correct thing to do is let SIGBUS kill the process. You can at least expect to handle SIGBUS for a file you've mapped.
  - tntn 8y ago
    So should we extend your conclusion above to the following? "There is probably enough evidence in this thread to use it as a reference for why typical apps should avoid virtual memory whenever possible -- it's clear almost nobody fully understands it" I'd suggest that is ludicrous, and for the same reason your original conclusion is also excessive.
    - _wmd 8y ago
      It is not constructive to form a sweeping generalization from a statement and then claim the sweeping generalization is ludicrous, implying the original statement is ludicrous. :) My first comment was in reply to one claiming anonymous memory did not have the same problems as file-backed memory, indicating the parent did not understand they are the same thing. The subsequent reply was to another comment continuing to claim anonymous memory was somehow safer, both instances supporting the notion that most people in this thread don't seem to understand mmap at all. What we're examining is a powerful (and consequently hazardous) OS feature that often provides only marginal performance improvement, yet introduces many exotic error paths into a program that have their own exotic problems (memory access in thread A can raise SEGV in thread B, async-signal safety), that 7 hours' commenting has not been sufficient to fully capture. This thread is full of upvoted miscomprehension, bad advice (spawn a child to deal with SEGV!?), obviously incorrect solutions (signalfd), and yet still manages to completely omit some critical characteristics of mmap, for example, that faults take a VM-global semaphore -- mmap can easily destroy multithreaded app performance in a way read() is immune to, because nobody expects file IO in one thread to cause malloc() latency in another. If this isn't evidence for "avoid this feature wherever possible", I really don't know what is.