3 ms·

i wonder the replication rate is for ML papers

• PaulHoule 1y ago
  From working in industry and rubbing shoulders with CS people who prioritize writing papers over writing working software I’m sure that in a high fraction of papers people didn’t implement the algorithm they thought they implemented.
  • avdelazeri 1y ago
    Don't get me started, I have seem repos that I'm fairly sure never ran in their presented form. A guy in our lab thinks authors purposefully mess up their code when publishing on GitHub to make it harder to replicate. I'm starting to come around on his theory.
• KingMob 1y ago
  And most medical studies. It's just as bad as social psych, if not worse, because there's real money at stake in churning out new drugs.
• intalentive 1y ago
  Nowadays everyone publishes their code. There’s typically a project page on github.io, a paper on arxiv.org, and a public repo.