3 ms·
The real cost driver with agents seems to be the repetitive context transmission since you re-send the history every step. I found I had to implement tiered mod
by storystarling 9mo ago
The real cost driver with agents seems to be the repetitive context transmission since you re-send the history every step. I found I had to implement tiered model routing or prompt caching just to make the unit economics work.