4 ms·
Why is the insane speed of 13KTPS of this site is not more on the the top of the AI conversations?
by kkotak 3mo ago
Why is the insane speed of 13KTPS of this site is not more on the the top of the AI conversations?
- Ey7NFZ3P0nzAe 3mo agoIt's pretty well known by now.
- chromadon 3mo agoI asked it for a block of C++ code and it hit 14,189 tok/s. I assume it cached someone else's session?
- fcsp 3mo agoNo - it's custom silicon https://news.ycombinator.com/item?id=48693490 https://news.ycombinator.com/item?id=48693490
- mlrtime 3mo agoBecause I just tested it and it took 3-4 clarifications before it actually gave a correct response vs gemini/google search. It's not great, but good. I'd rather wait 3x as long.
- mike_hearn 3mo agoBecause there's been nothing to discuss since their announcement. Their API access immediately closed due to overwhelming demand and they didn't fab newer models than Llama3 yet. Probably they will make bank selling to HFT for a while.