5 ms·

Reproducibility is a huge issue in academia and in the software engineering world. As an anecdote, we were trying to reproduce results for a c++ algorithm for

Reproducibility is a huge issue in academia and in the software engineering world. As an anecdote, we were trying to reproduce results for a c++ algorithm for which we had the source code, on the same machine, same compiler, only 2 years after its original creation. Impossible. Our best guess is that some os/hw patch / firmware update had an impact on the random number generation, but we never managed to figure out what exactly caused the discrepancies that we were observing.

Something1234 8y ago
Did you look at your version of libc? What version the original linked to?
- whatever1 8y ago
  It's been some time, but I am 99% sure that we were linking with the correct version of libc, because we never used the system GCC (in CentOS they provide the very old 4.8 version), so we explicitly specified the compiler and the libc that we wanted to use.
  - Something1234 8y ago
    That might not even matter, because if you think you're using one version of gcc, but you're using another. I experienced this multiple times over the summer with scl and gcc.
org3432 8y ago
The way some video game studios do this is simply put a whole computer into the archive with all the software on it, so all the hardware and software are captured in time.
- whatever1 8y ago
  Shouldn't containers or VMs solve these problems?
  - org3432 8y ago
    No since the hardware changes over time, such as the video and audio hardware, and VMs don't prioritize emulating those anyway.
    - smegger001 8y ago
      wouldn't that depend on which VM you are using?
      - emmelaich 8y ago
        I agree. They used to say there would never be an Amiga emulator because of all the custom chips it had. But there are quite a few Amiga emulators now.
- drawkbox 8y ago
  Video games built on consoles have this benefit in that they are locked to a hardware iteration, granted there may be some server communication now but historically arcades and consoles capture the hardware snapshot as well. I prefer PC gaming as consoles are essentially locked in time in hardware but that does lend better to archival purposes.
- adrianN 8y ago
  What's the expected lifetime of a computer in storage? I'd think that bitrot sets in fairly quickly on storage medium.
  - sli 8y ago
    That's data rot or disc rot. Bit rot is also known as software rot.
peter303 8y ago
My professor requires the software, data and makefiles to reproduce any figure in a published paper. Then the figure would be erased with software reproducing it. The motivation was kind of selfish. The professor would update his textbooks every 5 or 10 years and noted how hard it was to recover old figures.

Something1234 8y ago

Did you look at your version of libc? What version the original linked to?

whatever1 8y ago
It's been some time, but I am 99% sure that we were linking with the correct version of libc, because we never used the system GCC (in CentOS they provide the very old 4.8 version), so we explicitly specified the compiler and the libc that we wanted to use.
- Something1234 8y ago
  That might not even matter, because if you think you're using one version of gcc, but you're using another. I experienced this multiple times over the summer with scl and gcc.

org3432 8y ago

The way some video game studios do this is simply put a whole computer into the archive with all the software on it, so all the hardware and software are captured in time.

whatever1 8y ago
Shouldn't containers or VMs solve these problems?
- org3432 8y ago
  No since the hardware changes over time, such as the video and audio hardware, and VMs don't prioritize emulating those anyway.
  - smegger001 8y ago
    wouldn't that depend on which VM you are using?
    - emmelaich 8y ago
      I agree. They used to say there would never be an Amiga emulator because of all the custom chips it had. But there are quite a few Amiga emulators now.
drawkbox 8y ago
Video games built on consoles have this benefit in that they are locked to a hardware iteration, granted there may be some server communication now but historically arcades and consoles capture the hardware snapshot as well. I prefer PC gaming as consoles are essentially locked in time in hardware but that does lend better to archival purposes.
adrianN 8y ago
What's the expected lifetime of a computer in storage? I'd think that bitrot sets in fairly quickly on storage medium.
- sli 8y ago
  That's data rot or disc rot. Bit rot is also known as software rot.

peter303 8y ago

My professor requires the software, data and makefiles to reproduce any figure in a published paper. Then the figure would be erased with software reproducing it. The motivation was kind of selfish. The professor would update his textbooks every 5 or 10 years and noted how hard it was to recover old figures.