The comment you're looking for is here[1], where @jart does indeed seem to be echoing the claim that memory usage is reduced to surprising levels given the LLM

The comment you're looking for is here[1], where @jart does indeed seem to be echoing the claim that memory usage is reduced to surprising levels given the LLM architecture. Justine's original guess was that this is because the model is sparse (?) but in actuality it seems like memory usage is just being reported strangely by the system and that a model that takes up 20GB on disk still effectively needs 20GB of RAM in the end. [1] https://github.com/ggerganov/llama.cpp/discussions/638#discussioncomment-5492916 https://github.com/ggerganov/llama.cpp/discussions/638#discu...