3 ms·

I’ve been playing around with groq and GPT OSS which they run at 1000 TPS (20B) or 800 TPS (120B) and the speed feels quite magical. I haven’t tried cerebras’

I’ve been playing around with groq and GPT OSS which they run at 1000 TPS (20B) or 800 TPS (120B) and the speed feels quite magical. I haven’t tried cerebras’ 3000 TPS yet but I did try the demo of that 15,000 TPS model whose name escapes me right now. I’m not sure if it makes a meaningful difference for my actual work, but it sure is amazing to watch it generate a screen full of text in the blink of an eye. I do think it’s super useful for rubbing little validation checks like showing it a diff to ensure that the changes are on task, and being able to do those quicker really helps because it means you can do many focused checks without them getting in the way.

robberth 4mo ago
https://chatjimmy.ai/ https://chatjimmy.ai/ ?
- msdz 4mo ago
  AFAIK Taalas, the company behind this demo, still only have their initially "hardwarized" model available to test in ChatJimmy, which IIRC is a rather stupid Llama 3ish 8b. Don't get me wrong though, that demo is still incredibly impressive & makes me very much excited for the hardware-based model era (potentially) ahead. Once you've experienced those speeds, you really start to think about the whole class of things that becomes possible; massively parallel decode paths, extensive reasoning loops, etc…
  - hedgehog 4mo ago
    For scale though if three or four chips that size can replicate a Qwen 27B experience that'll be quite useful.
- dkersten 4mo ago
  That’s the one. The speed is incredible and fun to see, but the model is rather weak to the point where I’m not sure it’s particularly useful for most people.
ayewo 4mo ago
> I haven’t tried cerebras’ 3000 TPS yet but I did try the demo of that 15,000 TPS model whose name escapes me right now. You were likely thinking of AI accelerator startup Taalas. Previous HN discussion: https://news.ycombinator.com/item?id=47086181 https://news.ycombinator.com/item?id=47086181

robberth 4mo ago

https://chatjimmy.ai/ https://chatjimmy.ai/ ?

msdz 4mo ago
AFAIK Taalas, the company behind this demo, still only have their initially "hardwarized" model available to test in ChatJimmy, which IIRC is a rather stupid Llama 3ish 8b. Don't get me wrong though, that demo is still incredibly impressive & makes me very much excited for the hardware-based model era (potentially) ahead. Once you've experienced those speeds, you really start to think about the whole class of things that becomes possible; massively parallel decode paths, extensive reasoning loops, etc…
- hedgehog 4mo ago
  For scale though if three or four chips that size can replicate a Qwen 27B experience that'll be quite useful.
dkersten 4mo ago
That’s the one. The speed is incredible and fun to see, but the model is rather weak to the point where I’m not sure it’s particularly useful for most people.

ayewo 4mo ago

> I haven’t tried cerebras’ 3000 TPS yet but I did try the demo of that 15,000 TPS model whose name escapes me right now. You were likely thinking of AI accelerator startup Taalas. Previous HN discussion: https://news.ycombinator.com/item?id=47086181 https://news.ycombinator.com/item?id=47086181