3 ms·

I make my own automated benchmarks

• ummonk 11mo ago
  Is there a tool / website that makes this process easy?
  • mudkipdev 11mo ago
    I coded it bun and openrouter(dot)ai. I have an array of benchmarks, each benchmark has a grader (for example, checking if it equals a certain string or grade the answer automatically using another LLM). Then I save all results to a file and render the percentage correct to a graph