3 ms·
True, but that's still 1,500 inference cycles. Even without external API fees, the latency and compute burden seems huge. I don't see how the economics work the
by storystarling 9mo ago
True, but that's still 1,500 inference cycles. Even without external API fees, the latency and compute burden seems huge. I don't see how the economics work there without significant subsidies.
- darrinm 9mo agoFWIW many tool calls can be and often are made in one inference cycle.