4 ms·
I made a test [0] which runs several different configurations against coding tasks from easy to hard. There is a test which it has to pass. Because of temperatu
by dataviz1000 7mo ago
I made a test [0] which runs several different configurations against coding tasks from easy to hard. There is a test which it has to pass. Because of temperature, the number of tokens per one shot vary widely with all the different configurations include this one. However, across 30 tests, this does perform worse.
[0] https://github.com/adam-s/testing-claude-agent https://github.com/adam-s/testing-claude-agent
- btown 7mo agoThis is an amazing analysis! Thank you for running this :)