4 ms·
I think speed is actually going to be a bigger factor than cost. Even projects where “money is no object” often hit a wall with LLM response times. Sure you ca
by jbotdev 1mo ago
I think speed is actually going to be a bigger factor than cost. Even projects where “money is no object” often hit a wall with LLM response times.
Sure you can speed things up with parallel work under subagents, but as with parallelizing traditional computational tasks, there are diminishing gains.
I keep hearing people saying just change the way you work to trust long-running agents and multi-task more, because they’re too slow to work with interactively for many use cases. I think that’s painful in a world where we expect humans to still heavily guide and interact with agents for their day-to-day work.
- perching_aix 1mo agoGiven the 750 tok/sec GPT 5.6 Sol Ultrafast (via Cerebras), the many-1000 tok/sec Chinese models, and the 15000 tok/sec Taalas HC1, I think we're well on the way towards seeing that solved too. Combine the two, and yeah, wild ride incoming. What's especially bewildering to me is that translated back to raw bandwidth, even 15000 tok/sec is just like what, 75 KB/s? Extremely meager amounts of data, moving mountains. It's already kinda funny seeing LLMs throw out effort estimates in wall time terms. It's always some "hours, days, weeks" tier thing, when in reality, it's gone and done in minutes.
- cactusplant7374 1mo agoOne of my projects has an estimate of 3,000+ hours. It seems accurate. It has spent months working on it.
- perching_aix 1mo agoAround the clock?
- cactusplant7374 1mo agoYes, with a few exceptions. It's much easier now that the five hour limit has been removed from codex.
- wyre 1mo agoCan I ask what you are working on?
- cactusplant7374 1mo agoI'm working on emulators with AOT and JIT.
- newAccount2025 1mo agoHaha. Yeah. I somehow find Claude Code’s utter lack of time reasoning to be very fun. “The work we did two weeks ago…” oh friend that was this morning…
- jbstack 1mo ago> they’re too slow to work with interactively for many use cases This just demonstrates how much we already take for granted the LLMs that we have now. If you compare it to what we had before (hand the task off to a junior dev and wait for them to complete the work) then it doesn't seem slow at all.