3 ms·
For knowing mmap exists (i.e. soft page faults are intensely useful explicitly for this kind of thing)? Or am I reading this wrong?
by mhh__ 4y ago
For knowing mmap exists (i.e. soft page faults are intensely useful explicitly for this kind of thing)? Or am I reading this wrong?
- v64 4y agoDue to nuances of how data is split across the various model files, the implementation involved some non-trivial steps in order for everything to be allocated correctly. The changes are here [1]. [1] https://github.com/ggerganov/llama.cpp/commit/5b8023d935401072b73b63ea995aaae040d57b87 https://github.com/ggerganov/llama.cpp/commit/5b8023d9354010...