2 ms·
Just to make sure I'm understanding this correctly. This paper signals that the authors have found a way to run Llama 2 70B, but with 1/8th the VRAM requiremen
by SeanAnderson 3y ago
Just to make sure I'm understanding this correctly.
This paper signals that the authors have found a way to run Llama 2 70B, but with 1/8th the VRAM requirements as compared to the original model, right?
And the output is on-par with the original along some metrics (ArcE/PiQA), within 25% on others (Wiki/C4), and the trajectory of their progress hints that there's even more ground to gain in the future?
- samus 3y agoYour last paragraph is the key. Without their improvements, the quality tradeoff would have been hard to stomach.
- londons_explore 3y agoWhen considering quality, you should really compare not to the original model, but to smaller models which use the same vram/compute
- karmasimida 3y agoBut people use large model for better metrics, this defeats the purpose.