3 ms·
And I wonder how redacting them reduces latency, as it sure as hell doesn’t make the responses any faster and bandwidth isn’t the issue here.
by stingraycharles 7mo ago
And I wonder how redacting them reduces latency, as it sure as hell doesn’t make the responses any faster and bandwidth isn’t the issue here.
- sothatsit 7mo agoThey provide thinking summaries, so I assume they have to call Haiku or some other model to summarise the thinking blocks.
- stingraycharles 7mo agoThat’s not asynchronous? Wouldn’t it make more sense to disable those thinking summaries in those cases rather than hiding the thinking altogether?