4 ms·

Well, I wonder if this could force people to start publishing their code so others can replicate the results. This would help greatly with the kinds of issues t

Well, I wonder if this could force people to start publishing their code so others can replicate the results. This would help greatly with the kinds of issues that are being discussed: - It would be much harder to hide makeup results and cherry-picked seeds - Useless research would not be as easy to hide - The real-world impact of the research would be more valued. - Researchers would be forced to recheck their code for it to be presentable, which could be a net benefit in terms of finding mistakes - and so on.. I do imagine this could harm the number of replications of the research. However, I think there can be a net positive effect from such a policy. What do you think?

the_snooze 5y ago
I'm active in the security and privacy field, and some conferences have started "artifact reviews" [1][2], where authors can optionally submit the data and code that go along with their work. It's definitely an improvement, but the devil is in the details. Should these be mandatory? If they're mandatory, then what quality standards should artifacts meet? Who's ensuring those standards are followed? How should proprietary code and sensitive human-subjects research data be handled? There's also the question of what code should be made public: Is it just the analysis that produced the metrics on the paper (e.g., R scripts, Jupyter notebooks, etc.)? Or should it also include the data collection and pre-processing? How about the code that didn't produce immediately publishable results, but could be useful for future work? [1] https://petsymposium.org/artifacts.php https://petsymposium.org/artifacts.php [2] https://www.usenix.org/conference/usenixsecurity21/call-for-artifacts https://www.usenix.org/conference/usenixsecurity21/call-for-...
- josalhor 5y ago
  > where authors can optionally submit the data and code that go along with their work. So cool. I wish there were more steps in this direction. > Should these be mandatory? As an outsider, I would say no. However that should have a detrimental effect on the ability to get published if you are from an academic institution. I can totally see a gray area when it comes to non-academic publishers. > There's also the question of what code should be made public Shouldn't it be as much as possible? The way I see it is that the easier to replicate a paper, the more people can focus on its true utility. That is, by abstracting ourselves away over the details of the paper, what does this really contribute? From my point of view, publishing as much as possible maximizes the real-world impact of the research.
bonoboTP 5y ago
These discussions remind me of how there's a years long delay before these ideas first reach boards like HN and then popular magazines and the taxi driver. Yes, code releasing is a big talking point and most major groups release code. It doesn't solve the subtler problems,like claiming 1% improvements as a solid contribution, unequal hyperparameter tuning, cherry picking datasets, overclaiming the novelty, salami publishing, lacking ablations, not comparing against better methods etc etc. Releasing code is a step but not a panacea. At least it partially protects against fully made up numbers which is not nothing, but it's just scratching the surface. In many cases the code is not so easy to use and the original authors may not be willing to help you with reproducing it, ignoring emails and GitHub issues. If you can't reproduce it, they can always claim something. In one case I was told they rewrote the codebase in a new framework since they generated the results in the paper and there may be small discrepancies due to that. Apparently that was enough change that they were no longer the best in the benchmark comparison. But nobody cares at this point, the paper was published, they are on to their next project. People also have the attitude that it doesn't matter anymore, the field has moved on already, a retraction doesn't change much just hurts the author and the group. The vast majority of papers isn't used anywhere actually in practice, the field is moving so fast that any method can only remain on top for a couple of months. Also, I don't think AI really is worse at this. It's rather that we are more open about it because we have less to lose and professors don't have an iron grip, due to the existence of industry. In some of academia your whole future depends on whose ass you kiss, it's better in CS. It's a bit like sexism accusations directed at nerds, as if they particularly bad. The molochian cancer is everywhere. The less you see it may just indicate its more effectively hidden.
bumby 5y ago
Publishing the code may not be enough. Part of the problem is that replication of a study isn’t usually rewarded in the academic sphere because of the obsession with novelty within publications. Most researchers want to spend their finite time on something they find interesting and also may help their career. One way may be to create a movement to foster good replication studies that can be published and rewarded.
- unishark 5y ago
  Novelty is in the job description though. I think the problem, especially in AI/ML is the obsession with "State of the Art" algorithms. This can be gamed by making claims with numerical "experiments". If an idea had to stand on its own, and be interesting even if it may not make a better pattern-recognition product, most of them probably wouldn't make it. Indeed that's essentially the requirements presented by reviewers and editors: "prove we should read about this otherwise-uninteresting method by giving numerical test results where it wins".
  - bumby 5y ago
    I’m not stating novelty isn’t be part of the aim of research, I just think we need to be aware of the perverse incentives it creates when it becomes the sole underlying criteria by which merit is judged. I think there’s room for expanding the definition of what is good research to include replication of prior art because the field in general will be better off for it. I imagine novel research would still be more prized but it wouldn’t relegate replication to being considered a waste of time.
OminousWeapons 5y ago
> Well, I wonder if this could force people to start publishing their code so others can replicate the results. This would help greatly with the kinds of issues that are being discussed: The real solution to the problem is to set a rule that says if you use a lab's published code base AT ALL for new work, you must cite the people who produced it along with the PI of that lab. No exceptions; acknowledgements don't count. Labs (especially those outside of CS such as life sciences labs) view code as IP and they don't want to release their code bases for two reasons: First, they don't think they will be properly cited when people use it (valid concern). Second, they think that they will lose their monopoly on future publications using the code base (less valid concern). Mandatory citation would solve both of those problems. Until a rule like this is in place, most labs will fight you tooth and nail on releasing code.
miltondts 5y ago
Great news! There are journals that do this! E.g. https://academic.oup.com/gigascience https://academic.oup.com/gigascience They require the full means for replication to be open access, and reviewers take this very seriously! They will run your code, open github issues if they detect bugs etc.
wiz21c 5y ago
AI seems to me as quite experimental. Designing better benchmarks (that is less cheatable) will help. Publishing code will go against IP policies, that should be handled at the political level : do you research for profit or for knowledge ? But yeah, code adds transparency and a whole lot of cheats... The funny thing is that any well organized coder will automate report production, so it's not like it is hard to do...
ssivark 5y ago
In the steel-manned implementation, I would like the plots and tables to be auto-re-generated on publication, from submitted code. That would ensure that the results can have no “hidden state” and the paper cannot claim anything more than what their publicly shared implementation guarantees. This is par for the course in theoretical work; we should strive to apply the same standards of rigor to experimental work.

the_snooze 5y ago

I'm active in the security and privacy field, and some conferences have started "artifact reviews" [1][2], where authors can optionally submit the data and code that go along with their work. It's definitely an improvement, but the devil is in the details. Should these be mandatory? If they're mandatory, then what quality standards should artifacts meet? Who's ensuring those standards are followed? How should proprietary code and sensitive human-subjects research data be handled? There's also the question of what code should be made public: Is it just the analysis that produced the metrics on the paper (e.g., R scripts, Jupyter notebooks, etc.)? Or should it also include the data collection and pre-processing? How about the code that didn't produce immediately publishable results, but could be useful for future work? [1] https://petsymposium.org/artifacts.php https://petsymposium.org/artifacts.php [2] https://www.usenix.org/conference/usenixsecurity21/call-for-artifacts https://www.usenix.org/conference/usenixsecurity21/call-for-...

josalhor 5y ago
> where authors can optionally submit the data and code that go along with their work. So cool. I wish there were more steps in this direction. > Should these be mandatory? As an outsider, I would say no. However that should have a detrimental effect on the ability to get published if you are from an academic institution. I can totally see a gray area when it comes to non-academic publishers. > There's also the question of what code should be made public Shouldn't it be as much as possible? The way I see it is that the easier to replicate a paper, the more people can focus on its true utility. That is, by abstracting ourselves away over the details of the paper, what does this really contribute? From my point of view, publishing as much as possible maximizes the real-world impact of the research.

bonoboTP 5y ago

These discussions remind me of how there's a years long delay before these ideas first reach boards like HN and then popular magazines and the taxi driver. Yes, code releasing is a big talking point and most major groups release code. It doesn't solve the subtler problems,like claiming 1% improvements as a solid contribution, unequal hyperparameter tuning, cherry picking datasets, overclaiming the novelty, salami publishing, lacking ablations, not comparing against better methods etc etc. Releasing code is a step but not a panacea. At least it partially protects against fully made up numbers which is not nothing, but it's just scratching the surface. In many cases the code is not so easy to use and the original authors may not be willing to help you with reproducing it, ignoring emails and GitHub issues. If you can't reproduce it, they can always claim something. In one case I was told they rewrote the codebase in a new framework since they generated the results in the paper and there may be small discrepancies due to that. Apparently that was enough change that they were no longer the best in the benchmark comparison. But nobody cares at this point, the paper was published, they are on to their next project. People also have the attitude that it doesn't matter anymore, the field has moved on already, a retraction doesn't change much just hurts the author and the group. The vast majority of papers isn't used anywhere actually in practice, the field is moving so fast that any method can only remain on top for a couple of months. Also, I don't think AI really is worse at this. It's rather that we are more open about it because we have less to lose and professors don't have an iron grip, due to the existence of industry. In some of academia your whole future depends on whose ass you kiss, it's better in CS. It's a bit like sexism accusations directed at nerds, as if they particularly bad. The molochian cancer is everywhere. The less you see it may just indicate its more effectively hidden.

bumby 5y ago

Publishing the code may not be enough. Part of the problem is that replication of a study isn’t usually rewarded in the academic sphere because of the obsession with novelty within publications. Most researchers want to spend their finite time on something they find interesting and also may help their career. One way may be to create a movement to foster good replication studies that can be published and rewarded.

unishark 5y ago
Novelty is in the job description though. I think the problem, especially in AI/ML is the obsession with "State of the Art" algorithms. This can be gamed by making claims with numerical "experiments". If an idea had to stand on its own, and be interesting even if it may not make a better pattern-recognition product, most of them probably wouldn't make it. Indeed that's essentially the requirements presented by reviewers and editors: "prove we should read about this otherwise-uninteresting method by giving numerical test results where it wins".
- bumby 5y ago
  I’m not stating novelty isn’t be part of the aim of research, I just think we need to be aware of the perverse incentives it creates when it becomes the sole underlying criteria by which merit is judged. I think there’s room for expanding the definition of what is good research to include replication of prior art because the field in general will be better off for it. I imagine novel research would still be more prized but it wouldn’t relegate replication to being considered a waste of time.

OminousWeapons 5y ago

> Well, I wonder if this could force people to start publishing their code so others can replicate the results. This would help greatly with the kinds of issues that are being discussed: The real solution to the problem is to set a rule that says if you use a lab's published code base AT ALL for new work, you must cite the people who produced it along with the PI of that lab. No exceptions; acknowledgements don't count. Labs (especially those outside of CS such as life sciences labs) view code as IP and they don't want to release their code bases for two reasons: First, they don't think they will be properly cited when people use it (valid concern). Second, they think that they will lose their monopoly on future publications using the code base (less valid concern). Mandatory citation would solve both of those problems. Until a rule like this is in place, most labs will fight you tooth and nail on releasing code.

miltondts 5y ago

Great news! There are journals that do this! E.g. https://academic.oup.com/gigascience https://academic.oup.com/gigascience They require the full means for replication to be open access, and reviewers take this very seriously! They will run your code, open github issues if they detect bugs etc.

wiz21c 5y ago

AI seems to me as quite experimental. Designing better benchmarks (that is less cheatable) will help. Publishing code will go against IP policies, that should be handled at the political level : do you research for profit or for knowledge ? But yeah, code adds transparency and a whole lot of cheats... The funny thing is that any well organized coder will automate report production, so it's not like it is hard to do...

ssivark 5y ago

In the steel-manned implementation, I would like the plots and tables to be auto-re-generated on publication, from submitted code. That would ensure that the results can have no “hidden state” and the paper cannot claim anything more than what their publicly shared implementation guarantees. This is par for the course in theoretical work; we should strive to apply the same standards of rigor to experimental work.