4 ms·
Follow up: https://github.com/facebookresearch/llama/issues/79#issuecomment-1460464011 https://github.com/facebookresearch/llama/issues/79#issuecom... claims 65
by downvotetruth 4y ago
Follow up: https://github.com/facebookresearch/llama/issues/79#issuecomment-1460464011 https://github.com/facebookresearch/llama/issues/79#issuecom... claims 65B was able to fit in 128 GB by unsharding & merging weights into a single file instead of the multiple pth with 172Gb max swap file usage & appears to stream to GPU.