3 ms·
"wafer scale" inference using 44GB of straight up SRAM per chip does not exactly sound "smaller and cheaper" to me. Just optimized.
by striking 2y ago
"wafer scale" inference using 44GB of straight up SRAM per chip does not exactly sound "smaller and cheaper" to me. Just optimized.
- jedberg 2y agoCheaper on a per token basis.
- sanxiyn 2y agoDoubtful. SRAM is not cheap, and this is entirely about SRAM vs HBM.
- jedberg 2y agoThey list the price in this press release. So either they're taking a big loss or they're doing it cheaper per token.
- throwup238 2y agoIt wouldn't be the first time a manufacturer ignored capital amortization to post better numbers.
- sanxiyn 2y agoNot even necessarily a loss let alone big, since their comparison includes margin.
- striking 2y agoI accept that. Your original comment left me under the impression that this represented a shift closer to the edge (I still don't think the hardware is all that much smaller), but I'll agree this is cheaper per token under full utilization.