4 ms·
Microsoft absolutely runs their own models on their own hardware, at scale, and they have done so for years just like every other hyperscaler -- Project Brainwa
by aseipp 3y ago
Microsoft absolutely runs their own models on their own hardware, at scale, and they have done so for years just like every other hyperscaler -- Project Brainwave was first publicly talked about as far back as 2018. The generative LLM craze is a recent phenomenon in comparison. They are absolutely going to go all in on putting AI functionality in Bing, in Excel, in Windows, etc etc. To do that, you need hardware.
None of this is really strange. It also wasn't strange when Google announced H100 systems while also pushing TPUs they developed. Microsoft has Jensen on stage because customers of Microsoft Azure demand Nvidia products. Customers of Google Cloud demand Nvidia products. So, they provide them those products, because not providing them loses those customers. It's that simple. Everyone involved in these deals acknowledges this.
- buildbot 3y agoAwesome! Someone who knows about Brainwave!
- aseipp 3y agoYeah! Did/do you work on it? The original publications were good timing; I was working as a consultant on an FPGA-based ML accelerator at the time the original stuff was talked about, and I really enjoyed reading everything I could about Brainwave! Really neat project from both a system design perspective (e.g. heterogeneous compiler) to the choice of using and interconnecting FPGAs and integrating the network/software/ML stack (IIRC, there was a good paper on the overlay network they used to make those custom functions available on the global network fabric.) I'm guessing at this point the ASICs make a lot more economic sense, though. :)
- buildbot 3y agoSorta! Tangentially on the data science side - Brainwave pre-dates me being at MS though. It's a really cool project and I'm glad someone else also thinks so :) I was enamored with FPGAs from undergrad, one of the professors I did research with was a consultant on the original Catapult project, I thought it was the coolest idea. It was pretty awesome to be able to come full circle a fews years later.
- sargun 3y agoThey’ve been doing custom accelerators for Bing for a few years. See: project catapult (2012). It’s all on FPGA, but similar. In some ways, Microsoft was 10 years ahead, but they are terrible as an organization at proliferating research projects to production across multiple orgs.