4 ms·
With 64GB can you run the 70B size llama models well?
by narrator 2y ago
With 64GB can you run the 70B size llama models well?
- threecheese 2y agoI should have qualified the meaning of “works perfectly” :) No 70b for me, but I am able to experiment with many quantized models (and I am using a Llama successfully, latency isn’t terrible)
- credit_guy 2y agoNo, you can't. I have 128 GB and a 70B llamafile is unusable.