4 ms·

It's true for databases, but this usage is closer to how shared libraries are loaded, just with extenal data. Also just to show how buggy mmap is, it's disable

It's true for databases, but this usage is closer to how shared libraries are loaded, just with extenal data. Also just to show how buggy mmap is, it's disabled in SQLite by default for example: https://www.sqlite.org/mmap.html https://www.sqlite.org/mmap.html Even now the windows version is buggy, but the great thing is that after it's fixed in llama.cpp, the open source community can just copy the solution. I don't know of many interactive utilities that were known for fast startup time _because_ of the use of mmap, but now we have one, which means many more are coming :)

quotemstr 4y ago
Buggy? In what way?
- masklinn 4y ago
  The "But there are also disadvantages" of the linked pages provides some of the issues with mmap. This also matches burntsushi's experience with mmap in ripgrep: - depending on concurrent accesses mmap can just sigbus on you (e.g. if the mapped file is being truncated) - mmap simply does not work with virtual filesystems, and will blow up on large files on 32b systems (windows also has further limitations on mmaps) - depending on workload mmap may not be faster than regular reads and memory buffers, ripgrep will mmap when working on just a few files, but will use normal buffers for large file counts, because when you start reusing buffer you amortise allocation costs which you can't amortise when creating and destroying mappings - not only that but mmap/munmap are also globally blocking on the process, so in multithreaded processes it's very bad to map/unmap a lot, you can stall your own application - the semantics of mmap on crash are also somewhat risky, in that the OS will try very hard to sync, but that may not be desirable if the application crashes in the middle of a write
  - Jasper_ 4y ago
    LLAMA maps a single read-only file at startup. The only caveat that applies here is the virtual filesystems one, and it's a fair assessment, so a fallback might be warranted, but mmap seems fine by default.
  - smeenai 4y ago
    https://reviews.llvm.org/D69294 https://reviews.llvm.org/D69294 is an interesting case we ran into with LLD (the LLVM linker), where the the memory pressure from mmap'ing a large output file combined with filesystem compression resulted in particularly bad performance.
  - quotemstr 4y ago
    I wouldn't call any of these things a "bug", in the sense of a behavior that departs from an explicit or implicit specification. Some of them are just advantages and disadvantages of that overall approach. Also, I'm disappointed that VM languages don't turn SIGBUS accessing a memory mapped region into a VM exception. They could, and it wouldn't even be that hard.

quotemstr 4y ago

Buggy? In what way?

masklinn 4y ago
The "But there are also disadvantages" of the linked pages provides some of the issues with mmap. This also matches burntsushi's experience with mmap in ripgrep: - depending on concurrent accesses mmap can just sigbus on you (e.g. if the mapped file is being truncated) - mmap simply does not work with virtual filesystems, and will blow up on large files on 32b systems (windows also has further limitations on mmaps) - depending on workload mmap may not be faster than regular reads and memory buffers, ripgrep will mmap when working on just a few files, but will use normal buffers for large file counts, because when you start reusing buffer you amortise allocation costs which you can't amortise when creating and destroying mappings - not only that but mmap/munmap are also globally blocking on the process, so in multithreaded processes it's very bad to map/unmap a lot, you can stall your own application - the semantics of mmap on crash are also somewhat risky, in that the OS will try very hard to sync, but that may not be desirable if the application crashes in the middle of a write
- Jasper_ 4y ago
  LLAMA maps a single read-only file at startup. The only caveat that applies here is the virtual filesystems one, and it's a fair assessment, so a fallback might be warranted, but mmap seems fine by default.
- smeenai 4y ago
  https://reviews.llvm.org/D69294 https://reviews.llvm.org/D69294 is an interesting case we ran into with LLD (the LLVM linker), where the the memory pressure from mmap'ing a large output file combined with filesystem compression resulted in particularly bad performance.
- quotemstr 4y ago
  I wouldn't call any of these things a "bug", in the sense of a behavior that departs from an explicit or implicit specification. Some of them are just advantages and disadvantages of that overall approach. Also, I'm disappointed that VM languages don't turn SIGBUS accessing a memory mapped region into a VM exception. They could, and it wouldn't even be that hard.