4 ms·
This is frankly the best benchmark you can use on an LLM.
by poser-boy 3y ago
This is frankly the best benchmark you can use on an LLM.
- avereveard 3y agoYou can ask gpt4 or other high value model to rate two chat logs for coherency etc, not as accurate as human evaluation, but you don't have to read thousand lines of text if comparing many models.
- brucethemoose2 3y agoThis is problematic if you are comparing a model in the same base family as the evaluator, as it will probably favor itself because it literally has the sequences it would naturally emit.
- Breza 3y agoEspecially since most devs have a specific use case in mind. Coming up with tests that are tailored to your needs will always be more informative than off the shelf metrics.