3 ms·
As a former AI accelerator employee (laid off), I'm kind of happy I was laid off because I realistically don't see a need for specialized hardware anymore. Lar
by anon291 4y ago
As a former AI accelerator employee (laid off), I'm kind of happy I was laid off because I realistically don't see a need for specialized hardware anymore.
Large companies can afford Nvidia. Nvidia's software stack is best in class. There's no business need here and the model execution is increasingly becoming possible on single consumer GPUs.
The only place where I see specialized chips excelling is on the edge or if they are truly revolutionary (in which case they're only an acquisition target for Nvidia).
The truth is... The large language models are likely excessively large.
- foobiekr 4y agoPower is the main reason to do custom ASICs. I’d be curious as to your opinion of Recogni given they are claiming a 10x power reduction per unit compute.
- anon291 4y agoUnfortunately, I've worked at several players which promise power reductions. It doesn't matter though. People don't care about cost at this point. If you are cost-sensitive you're not doing the kind of revolutionary AI work these companies need to create a competitive moat. And once your model works on NVIDIA and is trained, how much are you going to spend on ML engineers to make it work on something else? Because that cost better be less than the marginal cost reduction on electricity. Plus, NVIDIA et al will likely get more and more efficient.
- bitL 4y agoCost sensitive folks post TopTal jobs such as "build a ChatGPT-like service from the scratch for this industry in 2 weeks for $25/hour".
- nl 4y agoThis is exactly right. The only exception is running things on mobile. There is demand for porting models to run natively on mobile, and somewhat reasonable support for doing this.