4 ms·

By sanity check you mean shows the numbers I wanted to see? Not saying there's necessarily something wrong in this particular code but I know first hands how li

By sanity check you mean shows the numbers I wanted to see? Not saying there's necessarily something wrong in this particular code but I know first hands how little effort is put into validating the scientific codes and how much of produces just random crap.

kspacewalk2 5y ago
Which is why releasing their code is critical. They lay it all out for you, with their pride/reputation on the line if it produces 'just random crap'. Writing exhaustive unit tests is probably not something they are good at, nor is it a requirement for them, nor is it the best use of their time. I too know first hand that students and even academics who aren't properly trained and given the right tooling will write prototype code that's not production ready. Occasionally there's even a bona fide mistake, though usually it's just very un-generalizable. It's part of my job to make some of it more production ready. That's the best use of my time and training. Conversely, I'm shit at generating useful research ideas or writing papers. Just don't have the right combination of intuition/training/experience. Good thing my academic institution has the resources to employ them and me.
epistasis 5y ago
How can you know which numbers you want to see? With tests it is often best practice to write the tests before writing the code that gets tested. But what if you don't know ahead of time, and can't possibly ever know ahead of time? Writing "tests" for this would be about adding stuff afterwards, like "the max and min values of this column were X and Y". But that is expected to break, if anything changes, because it's not testing anything useful. My question is: what is one concrete test here that would be useful and actually provide confidence that the code is doing what it should be doing, and how does that test provide better sanity checking than inspecting the tables at each step of the process?

kspacewalk2 5y ago

Which is why releasing their code is critical. They lay it all out for you, with their pride/reputation on the line if it produces 'just random crap'. Writing exhaustive unit tests is probably not something they are good at, nor is it a requirement for them, nor is it the best use of their time. I too know first hand that students and even academics who aren't properly trained and given the right tooling will write prototype code that's not production ready. Occasionally there's even a bona fide mistake, though usually it's just very un-generalizable. It's part of my job to make some of it more production ready. That's the best use of my time and training. Conversely, I'm shit at generating useful research ideas or writing papers. Just don't have the right combination of intuition/training/experience. Good thing my academic institution has the resources to employ them and me.

epistasis 5y ago

How can you know which numbers you want to see? With tests it is often best practice to write the tests before writing the code that gets tested. But what if you don't know ahead of time, and can't possibly ever know ahead of time? Writing "tests" for this would be about adding stuff afterwards, like "the max and min values of this column were X and Y". But that is expected to break, if anything changes, because it's not testing anything useful. My question is: what is one concrete test here that would be useful and actually provide confidence that the code is doing what it should be doing, and how does that test provide better sanity checking than inspecting the tables at each step of the process?