4 ms·
This is insanely fast, obviously a game changer over time. You should try the demo! This seems to be using custom inference-only HW. It makes a ton of sense to
by tboerstad 3y ago
This is insanely fast, obviously a game changer over time. You should try the demo!
This seems to be using custom inference-only HW. It makes a ton of sense to use different HW for inference vs training, the requirements are different.
Nvidia, as far as I can tell, is focusing all-in on training and hoping the same HW will be used for inference.
Exciting times!
- tome 3y agoHi there, I work for Groq. That's right. We love graphics processors for training but for inference our language processor (LPU) is by far the fastest and lowest latency. Feel free to ask me anything.
- artninja1988 3y agoAre they only available on the cloud? Are you planning on releasing a consumer version?
- tome 3y agoMostly available as a service via cloud API at the moment. The systems themselves are too big for consumers but we will sell systems to corporations.
- mayli 3y agoWhat's the scale of hardware behind this demo, in terms of watts, transistors and cost?