3 ms·
Important and unsurprising sentence from the paper's abstract: "A number of open source implementations of NTMs exist but are unstable during training and/or fa
by YorkshireSeason 8y ago
Important and unsurprising sentence from the paper's abstract:
"A number
of open source implementations of NTMs exist but are unstable during
training and/or fail to replicate the reported performance of NTMs"
- simias 8y agoWhy do you deem it unsurprising? Do you think that the opensource neural network implementations are subpar?
- YorkshireSeason 8y agoAnecdotally: I've tried to replicate some recent AI/ML papers and failed. So have some of my acquaintances.
- albertzeyer 8y agoI would expect that it was just as unstable for DeepMind. But they can automatically run lots of experiments with different hyper params so that is less an issue for them.
- nickpsecurity 8y agoSome have written about a "reproducibility crisis" in machine learning research: https://petewarden.com/2018/03/19/the-machine-learning-reproducibility-crisis/ https://petewarden.com/2018/03/19/the-machine-learning-repro...