3 ms·
Wow that's incredible. And legit too. I was reading through issues on llama-cpp about implementing memory swapping so I didn't think it had been done. Thanks!
by heyoni 3y ago
Wow that's incredible. And legit too. I was reading through issues on llama-cpp about implementing memory swapping so I didn't think it had been done.
Thanks!
- Filligree 3y agoIt’s really just a difference in accounting. Memory used for memory-mapped files aren’t shown in the “used” header, but instead the disk cache one. And doesn’t need to be swapped out to be discarded, so if you lack the memory it just slows everything down without an obvious cause.