3 ms·
Yeah, if they can fit an 8B model that's really good at improving the output by thinking, running at 16K tok/s on Taalas would be mind-blowing.
by pants2 4mo ago
Yeah, if they can fit an 8B model that's really good at improving the output by thinking, running at 16K tok/s on Taalas would be mind-blowing.
- le-mark 4mo agoGiven this and the quality of open models, it makes no sense to me that there’s a future for Anthropic et all?
- james_marks 4mo agoPackaging a capability into a consumable form will still be business. It's like web hosting; all the open source tools are there and free, and yet website tools, hosts, etc flourish.
- WhiteDawn 4mo agoIt’s true, but hosting prices are still within spitting distance of rolling it yourself. SOTA providers are expecting some level of margin. Companies everywhere have a tight eye on their AI bills right now. The motivation is there if the models get good enough, even if it’s more painful.
- CamperBob2 4mo agoExactly, it's like web hosting. And I don't see a lot of web hosts with twelve-digit valuations.
- anuramat 4mo agoit's not like anthropic will stop improving their models