4 ms·
Wildly fast inference. And current chips are 14nm so headroom to get a lot better.
by scosman 3y ago
Wildly fast inference. And current chips are 14nm so headroom to get a lot better.
- jsheard 3y agoNote that SRAM density doesn't scale at the same rate as logic density, and Groqs "secret sauce" is putting a ton of SRAM on their chips. Their stuff won't necessarily see the full benefits of switching to denser nodes if the bottleneck is how much SRAM they can pack onto each chip. https://www.tomshardware.com/news/no-sram-scaling-implies-on-more-expensive-cpus-and-gpus https://www.tomshardware.com/news/no-sram-scaling-implies-on... IIRC the last big jump for SRAM density was at 7nm, so they do still have that card to play, but progress has slowed to a crawl beyond that. TSMC 3nm SRAM is barely denser than TSMC 7nm SRAM.