3 ms·
An excerpt from the paper: "This means that developing these models would have cost around 2,638 MWh under our assumptions, and a total emission of 1,015 tCO2e
by brucethemoose2 4y ago
An excerpt from the paper:
"This means that developing these models would have cost around 2,638 MWh under our assumptions, and a total emission of 1,015 tCO2eq. We hope that releasing these models will help to reduce future carbon emission since the training is already done, and some of the models are relatively small and can be run on a single GPU."
Nowhere did I see actual inference VRAM consumption, but this is still very promising. Even if by a "single GPU" they mean an A100, hacks like DeepSpeed could still potentially squeeze the model onto desktop cards, laptop NPUs and such.