3 ms·

Enforcing higher standards is way more difficult than you think, even impractical most of the time. Having consensus and enforcing it is not trivial. Requiring

by yiyus 6y ago

Enforcing higher standards is way more difficult than you think, even impractical most of the time. Having consensus and enforcing it is not trivial. Requiring from every PhD student the level of quality of 3blue1brown is extremely naive. Your comment is the equivalent of saying there should be no armies and instead we should be peaceful and love each other. Great idea. And those excuses of why I cannot connect A to B? Let's just reach consensus to use the same connector everywhere! I'd like to live in that world, but unfortunately it is not the one we got.

altvali 6y ago
You're making a strawman here. What I want is that prestigious journals/conferences not accept papers unless they come with the code, dataset and weights required to replicate the result, and every formula come with explanations of the terms. If the terms are not widely used, a more thorough explanation should be requested by reviewers. It's not that difficult. There's already a Distill Prize for Clarity in Machine Learning. A great spark would be if a company like DeepMind or OpenAI would enforce standards like these internally and host a conference that rewards papers at that standard. It would be a great PR move, at great benefit for humanity.
- yiyus 6y ago
  There are many good papers that would not fulfill your requirements. I agree that ideally they would go through several revisions until every detail is fixed, but rejection levels would skyrocket. We would not have many great papers. I am not too familiar with the AI field, but I know that at least in my field the quality is very often too poor. We have to try to improve it. But if I rejected every paper I review because every detail to replicate it is not there, I would have rejected 99% of them. And some of the ones that eventually got accepted were very valid and very useful papers, and replication not so important after all.

by yiyus 6y ago

altvali 6y ago

You're making a strawman here. What I want is that prestigious journals/conferences not accept papers unless they come with the code, dataset and weights required to replicate the result, and every formula come with explanations of the terms. If the terms are not widely used, a more thorough explanation should be requested by reviewers. It's not that difficult. There's already a Distill Prize for Clarity in Machine Learning. A great spark would be if a company like DeepMind or OpenAI would enforce standards like these internally and host a conference that rewards papers at that standard. It would be a great PR move, at great benefit for humanity.

yiyus 6y ago
There are many good papers that would not fulfill your requirements. I agree that ideally they would go through several revisions until every detail is fixed, but rejection levels would skyrocket. We would not have many great papers. I am not too familiar with the AI field, but I know that at least in my field the quality is very often too poor. We have to try to improve it. But if I rejected every paper I review because every detail to replicate it is not there, I would have rejected 99% of them. And some of the ones that eventually got accepted were very valid and very useful papers, and replication not so important after all.