3 ms·
How do you actually measure the token savings? Do you compare the same task with and without the skill, or is it more of a noticeable difference over time?
by listingbott 27d ago
How do you actually measure the token savings? Do you compare the same task with and without the skill, or is it more of a noticeable difference over time?
- sejje 26d agoBy watching them get it the first time, instead of watching them hit 10 locations before finding it.
- mathgeek 26d agoBoth, in a way. In order to be sure it's the skill reducing token usage, you measure it with and without the skill, and then you do that periodically to account for all the other variables. Alternatively, you pick a belief based on prior evidence from either approach you mentioned, which is the natural thing that many of us do.
- dolmen 26d agoYou say: "you measure", "you do that", "you pick". But what about "I"?