3 ms·
I've seen reports of people being able to run LLMs at decent speeds on old Nvidia P40s. These are 24GB Pascal GPUs and can be bought for as low as $100 (althoug
by lhl 3y ago
I've seen reports of people being able to run LLMs at decent speeds on old Nvidia P40s. These are 24GB Pascal GPUs and can be bought for as low as $100 (although more commonly $200) on eBay.
- outofpaper 3y agoLink to report please
- lhl 3y agohttps://www.reddit.com/r/Oobabooga/comments/126dejd/comment/jebvlt0/?utm_source=reddit&utm_medium=web2x&context=3 https://www.reddit.com/r/Oobabooga/comments/126dejd/comment/...
- kiratp 3y agoYou can do it on CPU now. Benchmarks: https://github.com/ggerganov/llama.cpp/issues/34 https://github.com/ggerganov/llama.cpp/issues/34