Large LLM training workloads. It’s where the money’s at, and it’s why they went for "world’s biggest training run" headline rather than leading the pack on MLPe

Large LLM training workloads. It’s where the money’s at, and it’s why they went for "world’s biggest training run" headline rather than leading the pack on MLPerf standards. Not the least of which is because they can build their own LLM framework for customers to use, rather than spending time getting an MLPerf result that customers can’t possibly use. (I know because I’ve tried; it was quite a lot of archaeology to figure out how to dig out their 2019 MLPerf code for my own needs.) I don’t blame them. It’s the right move. I just miss the old days. But those aren’t coming back, and for better or worse the entire world is GPT and Diffusion focused for at least the next N years. Hopefully nerds can figure out how to scrape enough compute together to do interesting things during this gold rush period. It’s why TRC is that much more important, since it’s an opportunity to mint the next generation of engineers-turned-researchers. It would be smart for nvidia to offer something similar using their aging A100s. But I think only Google understands how powerful it is to capture the college students and interns.