2 ms·
Hear, hear. Even if the model fits, a few tokens per second make no sense. Time is money too.
by esafak 9mo ago
Hear, hear. Even if the model fits, a few tokens per second make no sense. Time is money too.
- tempoponet 9mo agoMaybe for a coding agent, but a daily/weekly report on sensitive info? If it were 2016 and this technology existed but only in 1 t/s, every company would find a way to extract the most leverage out of it.
- esafak 9mo agoBut it's 2026 and 'secure' (by executive standards) hosted options exist.
- dabockster 9mo ago> 'secure' (by executive standards) "Secure" in the sense that they can sue someone after the fact, instead of preventing data from leaking in the first place.
- michaellee8 9mo agoIf they figured out it can be this useful in 2016 running 1 t/s, they would make it run at least 20 t/s by 2019
- hex4def6 9mo agoIf I can start an agent and be able to walk away for 8 hours, and be confident it's 'smart' enough to complete a task unattended, that's still useful. At 3 tk/s, that's still 100-150 pages of a book, give or take.
- esafak 9mo agoTrue, that's still faster than a human, but they're not nearly that reliable yet.