4 ms·
The entirety of the coding evals are just 7 trivial coding tasks in Python? This is a joke.
by jacobgold 2mo ago
The entirety of the coding evals are just 7 trivial coding tasks in Python? This is a joke.
- ed-is-ai 1mo agoThanks for the feedback. It was limited to 7 to keep things balanced. On the premise most people don't just code. The 7 were an example of my realworld use cases - the point of this is to encourage people to run their own benchmarks and not just take what they read as gospel All the code and results are here for anyone who wants to delve, see if they can reproduce. That is the point of discourse https://github.com/ed-is-ai/featherbench https://github.com/ed-is-ai/featherbench