4 ms·
Reproducibility is a key element of the scientific process How is anyone else going to reproduce the experiment if it's going to cost them $10 million because
by c-linkage 2y ago
Reproducibility is a key element of the scientific process
How is anyone else going to reproduce the experiment if it's going to cost them $10 million because they don't work at Google and would have to rent the infrastructure?
- tokai 2y agoCheap compared to some high energy physic experiments.
- lostlogin 2y agoI was thinking this too. Splitting the atom, and various space program experiments would also be difficult to reproduce if someone wanted to try.
- rvnx 2y agoThis specific paper looks plausible, but a lot of published AI papers are simply fake because it is one of the sectors where it is possible to make non-reproducible claims. "We don't give source-code or dataset", but actually they didn't find or do anything of interest. It works and helps to get a salary raise or a better job, so they continue. A bit like when someone goes to a job interview, didn't do anything, and claims "My work is under NDA".
- Sebb767 2y agoBut what's the solution here? Not doing the (possibly) interesting research because it's hard to reproduce? That doesn't sound like a better situation. That being said, yes, this is hard to reproduce for your average Joe, but there are also a lot of companies (like OpenAI, Facebook, ...) that are able to throw this amount of hardware at the problem. And in a few years you'll probably be able to do it on commodity hardware.