3 ms·
Doesn't Phi-2 have testing data contamination, hence why it's performing well on these benchmarks? Most professionals in the field that I'm near would not touc
by tbalsam 3y ago
Doesn't Phi-2 have testing data contamination, hence why it's performing well on these benchmarks?
Most professionals in the field that I'm near would not touch that model with a 10 foot pole. We desperately need better validation/data contamination detection methods.
- triyambakam 3y agoCan you explain why you wouldn't want to use it, i.e. what data contamination is?
- 0xDEADFED5 3y agocontamination: training on the questions/answers used for LLM benchmarks. the top of the HuggingFace Open LLM leaderboard was pretty meaningless for a bit because many of the top scores were achieved by training on the evaluation data
- viraptor 3y agoI think you're thinking of phi-1.5 https://twitter.com/suchenzang/status/1701615026648605095 https://twitter.com/suchenzang/status/1701615026648605095 Unless there was something else for phi-2 as well?