3 ms·

One bad sign is that Google recently put out a press release that they’ve achieved the largest training job ever run, but they didn’t actually train any models.

One bad sign is that Google recently put out a press release that they’ve achieved the largest training job ever run, but they didn’t actually train any models. It was a bandwidth test: https://news.ycombinator.com/item?id=38227836 https://news.ycombinator.com/item?id=38227836 (Shoutout to DavidSJ who sleuthed this out based on the careful wording of the post.) This was a big shock to me since MLPerf is the gold standard for performance comparisons. The reason you want to train an actual model on real data is because it gives a precise idea of what you can expect in the field. Google’s 2019 MLPerf results are why I dove in to TPU land and fell in love with TPUs. Nowadays it’s all about language models and giant training runs and there’s not a lot of compute left over for average joes (which I very much was), but rants aren’t productive. What would be productive is for Google to focus on the basics: focus on MLPerf. It’s the standard benchmark for a reason. I can’t fault them for going where the money is, but it’s still sad to see the sorry state of Google being dead last. I’m still rooting for you, Jax team. I believe in you. TRC is looking better these days. If you need spare compute or just want to try out TPUs, give them a spin for a month. When I was complaining on Twitter in 2023 about TPUs being deleted after 30 minutes circa 2022, two people stepped in and said that they were happy with their quotas and haven’t observed similar pain points. So maybe they’ve solved the provisioning issues, and they’re starting to treat TRC members as more than third-class citizens again. But, especially in modern times, it’s hard to complain when compute is so scarce; you can do some pretty amazing things with TPUs if you focus on them for a few months. Thank you to the organizers of TRC (Jonathan in particular, their support lead, who is single handedly the most impressive support person I’ve ever seen by a huge margin; and Zak for keeping so many balls in the air while leading the TPU infra).

kccqzy 3y ago
> What would be productive is for Google to focus on the basics: focus on MLPerf. It’s the standard benchmark for a reason. I can’t fault them for going where the money is, but it’s still sad to see the sorry state of Google being dead last. You seem to be suggesting that Google isn't focusing on performance on MLPerf but on something else that's more lucrative. Do you want to expound on what that is?
- sillysaurusx 3y ago
  Large LLM training workloads. It’s where the money’s at, and it’s why they went for "world’s biggest training run" headline rather than leading the pack on MLPerf standards. Not the least of which is because they can build their own LLM framework for customers to use, rather than spending time getting an MLPerf result that customers can’t possibly use. (I know because I’ve tried; it was quite a lot of archaeology to figure out how to dig out their 2019 MLPerf code for my own needs.) I don’t blame them. It’s the right move. I just miss the old days. But those aren’t coming back, and for better or worse the entire world is GPT and Diffusion focused for at least the next N years. Hopefully nerds can figure out how to scrape enough compute together to do interesting things during this gold rush period. It’s why TRC is that much more important, since it’s an opportunity to mint the next generation of engineers-turned-researchers. It would be smart for nvidia to offer something similar using their aging A100s. But I think only Google understands how powerful it is to capture the college students and interns.
YetAnotherNick 3y ago
Lot of MLPerf tasks takes less than a minute. Meaning it likely takes more time starting the training and warming up than the actual training itself. MLPerf is widely outdated.
- deleted 3y ago
  [deleted]
varelse 3y ago
[dead]
ImJasonH 3y ago
Saving folks a googling: TRC is "TPU Research Cloud" (https://sites.research.google/trc/about/ https://sites.research.google/trc/about/)
brucethemoose2 3y ago
> Language models and giant training runs So... Its November 2023, and we have a handful of really good open source LLM base models (Mistral, Llama, MPT, Falcon, a few code models, a few Chinese ones like Yi, a Arabaic one) and a few notable proprietary ones, and a few cool vision models. ... Where is all that hoovered up accelerator compute going? Shouldnt we be swimming in open and closed LLMs by now?
- Palmik 3y ago
  Just a guess: (1) A lot of "me too" pre-training that should really be fine tuning; (2) lot of proprietary models whose weights will never be released (or even used :)).

kccqzy 3y ago

> What would be productive is for Google to focus on the basics: focus on MLPerf. It’s the standard benchmark for a reason. I can’t fault them for going where the money is, but it’s still sad to see the sorry state of Google being dead last. You seem to be suggesting that Google isn't focusing on performance on MLPerf but on something else that's more lucrative. Do you want to expound on what that is?

sillysaurusx 3y ago
Large LLM training workloads. It’s where the money’s at, and it’s why they went for "world’s biggest training run" headline rather than leading the pack on MLPerf standards. Not the least of which is because they can build their own LLM framework for customers to use, rather than spending time getting an MLPerf result that customers can’t possibly use. (I know because I’ve tried; it was quite a lot of archaeology to figure out how to dig out their 2019 MLPerf code for my own needs.) I don’t blame them. It’s the right move. I just miss the old days. But those aren’t coming back, and for better or worse the entire world is GPT and Diffusion focused for at least the next N years. Hopefully nerds can figure out how to scrape enough compute together to do interesting things during this gold rush period. It’s why TRC is that much more important, since it’s an opportunity to mint the next generation of engineers-turned-researchers. It would be smart for nvidia to offer something similar using their aging A100s. But I think only Google understands how powerful it is to capture the college students and interns.

YetAnotherNick 3y ago

Lot of MLPerf tasks takes less than a minute. Meaning it likely takes more time starting the training and warming up than the actual training itself. MLPerf is widely outdated.

deleted 3y ago
[deleted]

varelse 3y ago

[dead]

ImJasonH 3y ago

Saving folks a googling: TRC is "TPU Research Cloud" (https://sites.research.google/trc/about/ https://sites.research.google/trc/about/)

brucethemoose2 3y ago

> Language models and giant training runs So... Its November 2023, and we have a handful of really good open source LLM base models (Mistral, Llama, MPT, Falcon, a few code models, a few Chinese ones like Yi, a Arabaic one) and a few notable proprietary ones, and a few cool vision models. ... Where is all that hoovered up accelerator compute going? Shouldnt we be swimming in open and closed LLMs by now?

Palmik 3y ago
Just a guess: (1) A lot of "me too" pre-training that should really be fine tuning; (2) lot of proprietary models whose weights will never be released (or even used :)).