3 ms·
> The problem is that when you've reached terabytes of data and tens of cores you're not "ad-hoc" anymore. Not in any meaningful way, the code to explore a 50M
by throwaway894345 5y ago
> The problem is that when you've reached terabytes of data and tens of cores you're not "ad-hoc" anymore.
Not in any meaningful way, the code to explore a 50MB data set on a little machine looks the same as the code to explore a 50GB data set on a big machine, so of course one doesn’t need version control more than the other.
> Too often the math guys try to avoid responsibility by claiming they're doing "ad-hoc" work when they're clearly not anymore. It's convenient, yes, but leads to a bad place eventually.
This is a moralistic argument. The economic argument is that the primary artifact of research is insight, not code, so getting to that novel insight as fast as possible is paramount. Putting version control, tests, or other ceremony into the exploration loop is a pointless cost. Productionizing that insight can happen later in a more traditional software development workflow. It’s similar to how we write proofs of concept without the intensive testing effort that we would go into if we were writing production software.