3 ms·
According to Christopher Potts (Stanford Professor and Chair, Department of Linguistics, and Professor, by courtesy, Department of Computer Science), training a
by huijzer 4y ago
According to Christopher Potts (Stanford Professor and Chair, Department of Linguistics, and Professor, by courtesy, Department of Computer Science), training a large language model costs about 50 million [1].
[1]: https://youtu.be/-lnHHWRCDGk?t=637 https://youtu.be/-lnHHWRCDGk?t=637
- nl 4y agoYeah this is way wrong, unless counting the salaries of everyone involved for a few years in the lead up while writing software that ended up being used https://bigscience.huggingface.co/blog/bloom https://bigscience.huggingface.co/blog/bloom was trained with a $7M grant, and that was the first time they'd done it (so there was a lot of failures).