3 ms·
These are all facts coming from publicly available information, such as the Llama 3 paper and recent interviews. I'd dig up the specific timestamps, but I'm wor
by jiggawatts 3y ago
These are all facts coming from publicly available information, such as the Llama 3 paper and recent interviews. I'd dig up the specific timestamps, but I'm working at the moment and can't spend the next hour scrolling through long interviews.
The interviews are:
"Mark Zuckerberg - Llama 3, $10B Models, Caesar Augustus, & 1 GW Datacenters": https://www.youtube.com/watch?v=bc6uFV9CJGg https://www.youtube.com/watch?v=bc6uFV9CJGg
"Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI | Lex Fridman Podcast #416 ": https://www.youtube.com/watch?v=5t1vTLU7s40 https://www.youtube.com/watch?v=5t1vTLU7s40
The total FLOPS used in training is in bunch of places, I can't find a good link at the moment.