4 ms·

People are discovering mmap, it has been used for a very long time especially in databases.

by Thaxll 4y ago

xiphias2 4y ago
It's true for databases, but this usage is closer to how shared libraries are loaded, just with extenal data. Also just to show how buggy mmap is, it's disabled in SQLite by default for example: https://www.sqlite.org/mmap.html https://www.sqlite.org/mmap.html Even now the windows version is buggy, but the great thing is that after it's fixed in llama.cpp, the open source community can just copy the solution. I don't know of many interactive utilities that were known for fast startup time _because_ of the use of mmap, but now we have one, which means many more are coming :)
- quotemstr 4y ago
  Buggy? In what way?
  - masklinn 4y ago
    The "But there are also disadvantages" of the linked pages provides some of the issues with mmap. This also matches burntsushi's experience with mmap in ripgrep: - depending on concurrent accesses mmap can just sigbus on you (e.g. if the mapped file is being truncated) - mmap simply does not work with virtual filesystems, and will blow up on large files on 32b systems (windows also has further limitations on mmaps) - depending on workload mmap may not be faster than regular reads and memory buffers, ripgrep will mmap when working on just a few files, but will use normal buffers for large file counts, because when you start reusing buffer you amortise allocation costs which you can't amortise when creating and destroying mappings - not only that but mmap/munmap are also globally blocking on the process, so in multithreaded processes it's very bad to map/unmap a lot, you can stall your own application - the semantics of mmap on crash are also somewhat risky, in that the OS will try very hard to sync, but that may not be desirable if the application crashes in the middle of a write
    - Jasper_ 4y ago
      LLAMA maps a single read-only file at startup. The only caveat that applies here is the virtual filesystems one, and it's a fair assessment, so a fallback might be warranted, but mmap seems fine by default.
    - smeenai 4y ago
      https://reviews.llvm.org/D69294 https://reviews.llvm.org/D69294 is an interesting case we ran into with LLD (the LLVM linker), where the the memory pressure from mmap'ing a large output file combined with filesystem compression resulted in particularly bad performance.
    - quotemstr 4y ago
      I wouldn't call any of these things a "bug", in the sense of a behavior that departs from an explicit or implicit specification. Some of them are just advantages and disadvantages of that overall approach. Also, I'm disappointed that VM languages don't turn SIGBUS accessing a memory mapped region into a VM exception. They could, and it wouldn't even be that hard.
morelisp 4y ago
It has been used for so long in databases it should no longer be used in databases. http://www.cidrdb.org/cidr2022/papers/p13-crotty.pdf http://www.cidrdb.org/cidr2022/papers/p13-crotty.pdf
- rajnathani 4y ago
  Just saw your reply, we replied with the same CMU link :)
- deleted 4y ago
  [deleted]
rajnathani 4y ago
Databases usually avoid mmap in favor of their own mmap-like system: https://db.cs.cmu.edu/mmap-cidr2022/ https://db.cs.cmu.edu/mmap-cidr2022/
- dfox 4y ago
  That depends on exactly what kind of DBMS and underlying OS one is talking about. For DBMS that runs as one daemon or set of related daemons that has some kind of mechanism to share the afore mentioned buffer pool and synchronize acces to it then sure, it makes more sense, especially as you can do fine-grained copy on write in that bufferpool as an transaction isolation mechanism. For embedded databases that operate on shared file somewhere just mmap()ing that file allows you to use part of the same file as shared memory segment and for POSIX IPC and thus synchronization between otherwise unrelated processes that access the same data. One big caveat of this approach is that it depends on optional parts of POSIX that are realistically usable on Linux and Solaris and maybe some obscure systems (you can probably do the same thing on Windows NT), notably this does not work on essentially anything BSD-derived.
mftb 4y ago
They must use it in games too, no? For level loading, assets, etc...
- jbverschoor 4y ago
  In general game engines are almost full operating systems and reimplement many things such as memory management

People are discovering mmap, it has been used for a very long time especially in databases.

by Thaxll 4y ago

People are discovering mmap, it has been used for a very long time especially in databases.

It's true for databases, but this usage is closer to how shared libraries are loaded, just with extenal data. Also just to show how buggy mmap is, it's disabled in SQLite by default for example: https://www.sqlite.org/mmap.html https://www.sqlite.org/mmap.html Even now the windows version is buggy, but the great thing is that after it's fixed in llama.cpp, the open source community can just copy the solution. I don't know of many interactive utilities that were known for fast startup time _because_ of the use of mmap, but now we have one, which means many more are coming :)

quotemstr 4y ago
Buggy? In what way?
- masklinn 4y ago
  The "But there are also disadvantages" of the linked pages provides some of the issues with mmap. This also matches burntsushi's experience with mmap in ripgrep: - depending on concurrent accesses mmap can just sigbus on you (e.g. if the mapped file is being truncated) - mmap simply does not work with virtual filesystems, and will blow up on large files on 32b systems (windows also has further limitations on mmaps) - depending on workload mmap may not be faster than regular reads and memory buffers, ripgrep will mmap when working on just a few files, but will use normal buffers for large file counts, because when you start reusing buffer you amortise allocation costs which you can't amortise when creating and destroying mappings - not only that but mmap/munmap are also globally blocking on the process, so in multithreaded processes it's very bad to map/unmap a lot, you can stall your own application - the semantics of mmap on crash are also somewhat risky, in that the OS will try very hard to sync, but that may not be desirable if the application crashes in the middle of a write
  - Jasper_ 4y ago
    LLAMA maps a single read-only file at startup. The only caveat that applies here is the virtual filesystems one, and it's a fair assessment, so a fallback might be warranted, but mmap seems fine by default.
  - smeenai 4y ago
    https://reviews.llvm.org/D69294 https://reviews.llvm.org/D69294 is an interesting case we ran into with LLD (the LLVM linker), where the the memory pressure from mmap'ing a large output file combined with filesystem compression resulted in particularly bad performance.
  - quotemstr 4y ago
    I wouldn't call any of these things a "bug", in the sense of a behavior that departs from an explicit or implicit specification. Some of them are just advantages and disadvantages of that overall approach. Also, I'm disappointed that VM languages don't turn SIGBUS accessing a memory mapped region into a VM exception. They could, and it wouldn't even be that hard.

morelisp 4y ago

It has been used for so long in databases it should no longer be used in databases. http://www.cidrdb.org/cidr2022/papers/p13-crotty.pdf http://www.cidrdb.org/cidr2022/papers/p13-crotty.pdf

rajnathani 4y ago
Just saw your reply, we replied with the same CMU link :)
deleted 4y ago
[deleted]

rajnathani 4y ago

Databases usually avoid mmap in favor of their own mmap-like system: https://db.cs.cmu.edu/mmap-cidr2022/ https://db.cs.cmu.edu/mmap-cidr2022/

dfox 4y ago
That depends on exactly what kind of DBMS and underlying OS one is talking about. For DBMS that runs as one daemon or set of related daemons that has some kind of mechanism to share the afore mentioned buffer pool and synchronize acces to it then sure, it makes more sense, especially as you can do fine-grained copy on write in that bufferpool as an transaction isolation mechanism. For embedded databases that operate on shared file somewhere just mmap()ing that file allows you to use part of the same file as shared memory segment and for POSIX IPC and thus synchronization between otherwise unrelated processes that access the same data. One big caveat of this approach is that it depends on optional parts of POSIX that are realistically usable on Linux and Solaris and maybe some obscure systems (you can probably do the same thing on Windows NT), notably this does not work on essentially anything BSD-derived.

mftb 4y ago

They must use it in games too, no? For level loading, assets, etc...

jbverschoor 4y ago
In general game engines are almost full operating systems and reimplement many things such as memory management