2 ms·
There is a good metric in AI research: reproducible results on well known benchmarks (Big Bench for example).
by riku_iki 3y ago
There is a good metric in AI research: reproducible results on well known benchmarks (Big Bench for example).
- sebzim4500 3y agoThere's no good benchmark that tests for most of the issues with LLMs, for example. Maybe if we had a benchmark that told you how often a model hallucinated we would have solved the problem by now.
- riku_iki 3y agoThere are plenty of benchmarks with factual questions.