4 ms·

I assume given the architecture that Groq actually doesn't get any faster for batch sizes >1 I guess if you don't have any extra junk you can pack more process

I assume given the architecture that Groq actually doesn't get any faster for batch sizes >1 I guess if you don't have any extra junk you can pack more processing into the chip?

foundval 3y ago
(Groq Employee) Yes! Determinism + Simplicity are superpowers for ALU and interconnect utilization rates. This system is powered by 14nm chips, and even the interconnects aren't best in class. We're just that much better at squeezing tokens out of transistors and optic cables than GPUs are - and you can imagine the implications on Watt/Token. Anyways.. wait until you see our 4nm. :)