3 ms·
You're making a strawman here. What I want is that prestigious journals/conferences not accept papers unless they come with the code, dataset and weights requir
by altvali 6y ago
You're making a strawman here. What I want is that prestigious journals/conferences not accept papers unless they come with the code, dataset and weights required to replicate the result, and every formula come with explanations of the terms. If the terms are not widely used, a more thorough explanation should be requested by reviewers.
It's not that difficult. There's already a Distill Prize for Clarity in Machine Learning. A great spark would be if a company like DeepMind or OpenAI would enforce standards like these internally and host a conference that rewards papers at that standard. It would be a great PR move, at great benefit for humanity.
- yiyus 6y agoThere are many good papers that would not fulfill your requirements. I agree that ideally they would go through several revisions until every detail is fixed, but rejection levels would skyrocket. We would not have many great papers. I am not too familiar with the AI field, but I know that at least in my field the quality is very often too poor. We have to try to improve it. But if I rejected every paper I review because every detail to replicate it is not there, I would have rejected 99% of them. And some of the ones that eventually got accepted were very valid and very useful papers, and replication not so important after all.