4 ms·

There's a spectrum of agreement. (1) I want builds to be 100% reproducible on deployment servers and on CI/CD pipelines. Otherwise, you can undebuggable Heisen

by wegs 6y ago

There's a spectrum of agreement. (1) I want builds to be 100% reproducible on deployment servers and on CI/CD pipelines. Otherwise, you can undebuggable Heisenbugs. On the other hand, I don't want builds to be reproducible between developer machines. If I'm running Python 3.6 on Ubuntu, and another developer is running Python 3.7 on a Mac, and we have slightly different versions of numpy, that makes sure the system is not too brittle. Come to think of it, if I had infinite resources, I'd have several build machines with different (reproducible) configurations. (2) I'm a lot more spartan about dependencies than other developers I've met. (3) I'd never use apt to manage Python packages myself in something I'm working on. The constraint is in the other direction. If I build a tool, a user ought to be able to install it using apt in some future version of Debian, and likewise for other systems. Even if that's an abstract user. I've found that if I develop this way, the upsides outweigh the downsides, especially over extended periods. A lot of software gets built like a system which can only live in one places. There's a set of AWS machines, code on them, and that's the system. There might be a few copies of it (stage+dev+etc.), but you can't move it somewhere else. I like systems I build to be portable. Someone can bring them up-and-running on their own machine, ideally in a few minutes. I've always found that to be cheaper, in the long term.

nilkn 6y ago
I feel like your idea in (1) is not that unachievable with finite resources. It depends how far you took it but requiring tests to pass in a few mild perturbations of the target environment wouldn’t be that expensive in a lot of cases and not even that hard to set up. Sounds like this deserves a name like “perturbative testing” to me if it doesn’t already have one.
- wegs 6y ago
  In abstract, it doesn't take a huge amount of time and resources to do that. But, there are probably around a hundred higher-priority ideas achievable with the same resources which would take priority over this on the projects I'm working on right now. On projects I've worked on before, I think this would have made sense /technically/, given project priorities, but so did many other things which weren't done. It's a lot easier to make the case for resources for customer-facing features than for technical debt or infrastructure. So there's the political component too, which varies organization-by-organization. This is already done in a lot of projects with hardware. The Linux kernel will run on a thousand hardware and software configurations before integrating features. If I did this, I'd probably want at least three builds: * my pinned deployment versions (sometimes a release or two behind, sometimes bleeding-edge) * latest released version; and * HEAD If an upstream project is introducing a breaking change, I'd know immediately. That'd be super-helpful, probably both to me and to those projects. Come to think of it, the right way to do this might be to have three virtualenvs on my local machine, rather than just different targets in CI/CD....

by wegs 6y ago

nilkn 6y ago

I feel like your idea in (1) is not that unachievable with finite resources. It depends how far you took it but requiring tests to pass in a few mild perturbations of the target environment wouldn’t be that expensive in a lot of cases and not even that hard to set up. Sounds like this deserves a name like “perturbative testing” to me if it doesn’t already have one.

wegs 6y ago
In abstract, it doesn't take a huge amount of time and resources to do that. But, there are probably around a hundred higher-priority ideas achievable with the same resources which would take priority over this on the projects I'm working on right now. On projects I've worked on before, I think this would have made sense /technically/, given project priorities, but so did many other things which weren't done. It's a lot easier to make the case for resources for customer-facing features than for technical debt or infrastructure. So there's the political component too, which varies organization-by-organization. This is already done in a lot of projects with hardware. The Linux kernel will run on a thousand hardware and software configurations before integrating features. If I did this, I'd probably want at least three builds: * my pinned deployment versions (sometimes a release or two behind, sometimes bleeding-edge) * latest released version; and * HEAD If an upstream project is introducing a breaking change, I'd know immediately. That'd be super-helpful, probably both to me and to those projects. Come to think of it, the right way to do this might be to have three virtualenvs on my local machine, rather than just different targets in CI/CD....