3 ms·
>4x improvement on geospatial tasks with map in the loop. The graph shows a baseline 2% task success rate improving to to 8% task success rate, but the evals s
by OtherShrezzing 3mo ago
>4x improvement on geospatial tasks with map in the loop.
The graph shows a baseline 2% task success rate improving to to 8% task success rate, but the evals section details 100% success rates across the board.
I'm not sure what the effectiveness of this skill is from the readme. Is it 8% success, or 100% success?
- delfrrr 3mo agoIt's not % it's number of evals. Here is how I did evaluation https://dekart.xyz/blog/how-to-evaluate-claude-skill-output-quality-for-prompt-to-sql-scenarios/ https://dekart.xyz/blog/how-to-evaluate-claude-skill-output-...