4 ms·
It matters on consumer hardware since barely any model runs at reasonable speed.
by esafak 5mo ago
It matters on consumer hardware since barely any model runs at reasonable speed.
- gaeld 5mo agoIt also matters for thinking models and for agentic workflows, especially in software engineering, where a lot of tokens need to be output in iterative loops before the user sees any result. This is our main use case.