3 ms·
Yes, you are way off, because Groq doesn't make open source models. Groq makes innovative AI accelerator chips that are significantly faster than Nvidia's.
by ilaksh 10mo ago
Yes, you are way off, because Groq doesn't make open source models. Groq makes innovative AI accelerator chips that are significantly faster than Nvidia's.
- zamalek 10mo ago> Groq makes innovative AI accelerator chips that are significantly faster than Nvidia's. Yeah I'm disappointed by this, this is clearly to move them out of the market. Still, that leaves a vacuum for someone else to fill. I was extremely impressed by Groq last I messed about with it, the inference speed was bonkers.
- wmf 10mo agoIf it's that good Nvidia can just keep selling it.
- allovertheworld 10mo agomore like now Nvidia wants to release their own ASIC to combat google
- fragmede 10mo agoUmm... no one tell them, okay?
- LoganDark 10mo agoFor inference, but yes. Many hundreds of tokens per second of output is the norm, in my experience. I don't recall the prompt processing figures but I think it was somewhere in the low hundreds of tokens per second (so slightly slower than inference).