3 ms·
How are you evaluating LLM answers are right or wrong? Because I saw some wrong answers that were right and potentially right that were wrong. Are you just look
by TuringNYC 3y ago
How are you evaluating LLM answers are right or wrong? Because I saw some wrong answers that were right and potentially right that were wrong. Are you just looking for keywords, etc? Or is this all run beforehand and graded by humans?
- IAmGraydon 3y agoYeah this is completely broken judging from my experience. Often GPT would get the answer wrong and the site would claim that it was correct.
- TuringNYC 3y agoAnyone know a good method to judge right/wrong answers from LLMs? I can see keyword solutions to be brittle. Perhaps another LLM?