3 ms·

Let's try and simplify things here: the post we're both currently commenting on relates to doing file IO via nmap. Of course 'avoiding all use of virtual memory

by _wmd 7y ago

Let's try and simplify things here: the post we're both currently commenting on relates to doing file IO via nmap. Of course 'avoiding all use of virtual memory' is ridiculous, but nobody except you is suggesting that, and you continue to suggest it even after a long reply. The "powerful (and consequently hazardous) OS feature" here is using mmap for general file IO, it: - introduces resources leaks many developers can't profile - introduces VM bottlenecks 99% of developers can't profile - introduces random segfaults delivered to arbitrary threads in the running process, leading to crashes many developers can't diagnose - the mmap() interface itself is fundamentally unsafe in that it allows partially overwriting random bits of VM (MAP_FIXED) with file views, and worse still, allows those mappings to be read-write Once again, nobody has ever suggested avoiding virtual memory except you -- once again, that is impossible in a modern environment, but it is more than possible, and ultimately incredibly sensible, not to mention entirely on topic with regards to this thread and the article it is attached to, to suggest avoiding use of mmap for general file IO

klodolph 7y ago
- What resource leaks are introduced by mmap? - Nobody ever said mmap was always faster than the alternative. If you care about performance then you should do whole-application performance testing with and without features enabled (like mmap IO). This is not unusual, there are plenty of aspects of performance that are counter-intuitive, where speeding up one part of your program causes a seemingly unrelated part of your part of your program to slow down. - The signals are SIGBUS, and they can be intercepted, mapped with zeroes, and the errors can be propagated back to your app code later. This is not trivial but neither is it outrageous. - You can overwrite arbitrary memory with read(), too, you just have to pass it a pointer to something you want to overwrite. mmap() is not any less safe. Recall that typical use of MAP_FIXED is so you can overwrite an existing mmap() region with something else, not so you can nuke random parts of your address space. Keep in mind that you are, if nothing else, an indirect user of mmap(). The question is whether using mmap() directly is advantageous for your applicaiton. "Yes" is not an unreasonable nor outrageous answer for some applications.
a1369209993 7y ago
> nobody except you is suggesting [avoiding all use of virtual memory] https://news.ycombinator.com/item?id=19807322 https://news.ycombinator.com/item?id=19807322 > Anonymous mappings > typical apps should avoid mmap whenever possible All virtual memory is either a user-mode wrapper around mmap, or sbrk (which is functionally a kernel-mode wrapper around mmap).
- _wmd 7y ago
  This simply won't die, will it? I mean, while we're at it, let's advocate abandonment of all higher level languages because essentially they all boil down to machine code, and nobody could recommend working directly with machine code any more, could they. (But that would be a sweeping generalization)
  - tntn 7y ago
    > This simply won't die, will it? Please clarify how you think that statement is incorrect. As far as I'm concerned, it won't die because that's how virtual memory works, and you have something going on in your head that is either wrong or massive hairsplitting. I would have expected a better understanding from someone bloviating about how the try of the commenters are too thick to understand mmap and virtual memory.
tntn 7y ago
> here is using mmap for general file IO > nobody has ever suggested avoiding virtual memory except you Do you know what the "anonymous" in "anonymous mapping" means? You are the one that started asserting that anonymous mapping from mmap have the same difficulties as mapping of normal files and therefore too dangerous to use. https://news.ycombinator.com/item?id=19807322 https://news.ycombinator.com/item?id=19807322
- _wmd 7y ago
  I hate to do this, but: > paragraph, n.: a distinct section of a piece of writing, usually dealing with a single theme and indicated by a new line, indentation, or numbering In the original comment you will find two of these, the former correcting an error in the parent comment, the latter making an observation based on the obvious brainwrong riddled throughout this thread I'm done replying, you're of course free to continue checking in hazardous and suspect file IO code, as the rest of us are free to giggle at such things before ripping them out

by _wmd 7y ago

klodolph 7y ago

- What resource leaks are introduced by mmap? - Nobody ever said mmap was always faster than the alternative. If you care about performance then you should do whole-application performance testing with and without features enabled (like mmap IO). This is not unusual, there are plenty of aspects of performance that are counter-intuitive, where speeding up one part of your program causes a seemingly unrelated part of your part of your program to slow down. - The signals are SIGBUS, and they can be intercepted, mapped with zeroes, and the errors can be propagated back to your app code later. This is not trivial but neither is it outrageous. - You can overwrite arbitrary memory with read(), too, you just have to pass it a pointer to something you want to overwrite. mmap() is not any less safe. Recall that typical use of MAP_FIXED is so you can overwrite an existing mmap() region with something else, not so you can nuke random parts of your address space. Keep in mind that you are, if nothing else, an indirect user of mmap(). The question is whether using mmap() directly is advantageous for your applicaiton. "Yes" is not an unreasonable nor outrageous answer for some applications.

a1369209993 7y ago

> nobody except you is suggesting [avoiding all use of virtual memory] https://news.ycombinator.com/item?id=19807322 https://news.ycombinator.com/item?id=19807322 > Anonymous mappings > typical apps should avoid mmap whenever possible All virtual memory is either a user-mode wrapper around mmap, or sbrk (which is functionally a kernel-mode wrapper around mmap).

_wmd 7y ago
This simply won't die, will it? I mean, while we're at it, let's advocate abandonment of all higher level languages because essentially they all boil down to machine code, and nobody could recommend working directly with machine code any more, could they. (But that would be a sweeping generalization)
- tntn 7y ago
  > This simply won't die, will it? Please clarify how you think that statement is incorrect. As far as I'm concerned, it won't die because that's how virtual memory works, and you have something going on in your head that is either wrong or massive hairsplitting. I would have expected a better understanding from someone bloviating about how the try of the commenters are too thick to understand mmap and virtual memory.

tntn 7y ago

> here is using mmap for general file IO > nobody has ever suggested avoiding virtual memory except you Do you know what the "anonymous" in "anonymous mapping" means? You are the one that started asserting that anonymous mapping from mmap have the same difficulties as mapping of normal files and therefore too dangerous to use. https://news.ycombinator.com/item?id=19807322 https://news.ycombinator.com/item?id=19807322

_wmd 7y ago
I hate to do this, but: > paragraph, n.: a distinct section of a piece of writing, usually dealing with a single theme and indicated by a new line, indentation, or numbering In the original comment you will find two of these, the former correcting an error in the parent comment, the latter making an observation based on the obvious brainwrong riddled throughout this thread I'm done replying, you're of course free to continue checking in hazardous and suspect file IO code, as the rest of us are free to giggle at such things before ripping them out