3 ms·
> The name of the game has been custom SoCs and ASICs for a couple years now, because inference and model training is an "embarrassingly parallel" problem, and
by philipwhiuk 1y ago
> The name of the game has been custom SoCs and ASICs for a couple years now, because inference and model training is an "embarrassingly parallel" problem, and models that are optimized for older hardware can provide similar gains to models that are run on unoptimized but more performant hardware.
Is anyone else getting crypto flashbacks?
- kcb 1y agoThe difference is crypto wasn't memory and throughput dependent. That's why a small asic on a USB stick could outperform a GPU.
- alephnerd 1y agoOne thing to point out - "ASIC" is more of a business term than a technical term. The teams that work on custom ASIC design at (eg.) Broadcom for Microsoft are basically designing custom GPUs for MS, but these will only meet the requirements that Microsoft lays out, and Microsoft would have full insight and visibility into the entire architecture.