3 ms·
> hardware takes a couple generations to become a serious contender Not really and for the same reason Chinese players like Biren are leapfrogging - much of th
by alephnerd 1y ago
> hardware takes a couple generations to become a serious contender
Not really and for the same reason Chinese players like Biren are leapfrogging - much of the workload profile in AI/ML is "embarrassingly parallel", thus reducing the need for individual ASICs to be bleeding edge performant.
If you are able to negotiate competitive fabrication and energy supply deals, you can mass produce your way into providing "good enough" performance.
Finally, the persona who cares about hardware performance in training isn't in the market for cloud offered services.
- kenjackson 1y agoAnd current LLM architectures affinitize differently to HW than DNNs even just a decade ago. If you have the money and technical expertise (both of which I assume MS has access to) then a late start might actually be beneficial.
- rcxdude 1y agoAs I understood it the main bottleneck is interconnects, anyhow. It's more difficult to keep the ALUs fed than it is to make them fast enough, especially once your model can't fit in one die/PCB. And that's in principle a much trickier part of the design, so I don't really know how that shakes out (is there a good enough design that you can just buy as a block?)
- deleted 1y ago[deleted]
- alephnerd 1y agoThe bet I'm seeing is to try and invest in custom ASICs to become integrated as part of an SoC to solve that interconnect bottleneck. It's largely a solved problem based on Google/Broadcom's TPU work - almost everyone is working with Broadcom to design their own custom ASIC and SoC.