4 ms·
>"The replication crisis is largely concerned with known problems, such as the lack of replication standards, non-availability of data, or p-hacking. One hither
by nonbel 8y ago
>"The replication crisis is largely concerned with known problems, such as the lack of replication standards, non-availability of data, or p-hacking. One hitherto unknown problem is the potential for software companies’ changes to the algorithms used for calculations to cause discrepancies between two sets of reported results."
- June 7th, 2018
They didnt know about this?
- s-shellfish 8y agoNo, they expect developers to know everything, apparently. At least to know everything that doesn't give them a dopamine hit when two result sets match, validating their theories/beliefs. This is what developers deal with all the time, so this is obvious. Code should work this way, but it doesn't always. Yes, we can all make heroes of ourselves by thinking of ourselves as saviors for those edge cases of catastrophic failure, but we don't, because we are aware, that bugs happen - they are a mathematical certainty assuming one doesn't have access to all information, including every single use case of a program in the future.
- nonbel 8y agoI mean almost the first thing that happened to me when I tried to use commercial stats software was being unable to reproduce a calculation manually. Eventually found out they had a known bug in their code that was left in for ~20 years and when they fixed it they called it the "enhanced version". The calculation was based on the original version of a paper that was retracted and corrected. I would never trust commercial stats software after that. I really mean this was close to the very first thing, so I would assume it has to be very common.