4 ms·
I don't mean evaluating a model at the end of training, I mean datasets used to quantify the relative performance across models and algorithms. Copyrighted dat
by knaik94 3y ago
I don't mean evaluating a model at the end of training, I mean datasets used to quantify the relative performance across models and algorithms.
Copyrighted data would make it harder to share and use, to reproduce results. I don't remember coming across a benchmark dataset that was copyrighted.
- bjourne 3y agoThat's very odd because I can't for the life of me think of an ml benchmark dataset that is not copyrighted. Copyrighted datasets are everywhere in ml research.
- knaik94 3y agoJust looking at the eval datasets used in the HF leaderboard, ARC is CC BY-SA, TruthfulQA is Apache, and HellaSwag and MMLU are both MIT. The first dataset many people will explore ml with, MNIST, is GPL-3.0 license. The audio courpus datasets I've used have all been similarly permissively licensed, for example LibriTTS is CC BY 4.0. LJ Speech, one of the main datasets used in Tacotron is public domain. Maybe I should have been more explicit and used the term permissively copyrighted instead to help avoid confusion. https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...