3 ms·
Meta unveils a new large language model that can run on a single GPU
- elpookie 4y agoI hope more development goes into these smaller language models that are still usable but which I can be run on my own pc.
- brucethemoose2 4y agoAn excerpt from the paper: "This means that developing these models would have cost around 2,638 MWh under our assumptions, and a total emission of 1,015 tCO2eq. We hope that releasing these models will help to reduce future carbon emission since the training is already done, and some of the models are relatively small and can be run on a single GPU." Nowhere did I see actual inference VRAM consumption, but this is still very promising. Even if by a "single GPU" they mean an A100, hacks like DeepSpeed could still potentially squeeze the model onto desktop cards, laptop NPUs and such.