5 ms·

I can't stand builds that just reach out to the internet for things.

lanstin 5y ago
Yeah seems like basic hygiene especially given the supply chain attacks but also a lost cause. No one has the skills to do builds without the internet. Even forcing teams to use an allow list for the internet involving fighting a lot of angry people.
- contingencies 5y ago
  One of the few decent uses for containers is to enforce proxied internet so build process artifacts can be auto-stored for subsequent builds. For the worst offender I am aware of, try building a flutter project... it silent internet gets artifacts from at least three different packaging systems (node, cocoapods, android packages), all of which have caused hard-to-debug failures.
duped 5y ago
How do you build software in 2021 then? Do you write everything from scratch?
- spoils19 5y ago
  ... yes? I've often been ridiculed for wanting to build everything in-house, but events like these just validate that sometimes, to get the most reliable and best software, you'll have to reinvent the wheel a few times.
  - duped 5y ago
    To be honest that's deserving of ridicule. Reinventing wheels is how you introduce fragility and instability, not prevent it.
- still_grokking 5y ago
  One can use the packages provided by the distribution. That's how it worked since forever for things like C/C++. For example distris like Debian even mandate that you build software for them only relying on things that are already packaged. No arbitrary internet downloads allowed during build. The build environment contains only what is provided by the stated build-dependencies of your package, which are themself of course Debian packages, and some common minimal build base (like shell and some utils).
  - duped 5y ago
    What if the package you want isn't provided by the distribution? What if the distribution's version doesn't support the features you need? What if it can't be built by relying only on things that are already packaged? How do you distribute your software so it runs on other distributions? Do you maintain a different build for each package manager? What if you want to run on platforms that don't have standard package managers, like MacOS or Windows? How do you update the packages without internet downloads during provisioning the build? The C/C++ model has proven to be fatally flawed. It hasn't worked, that's why modern languages eschew reliance on the distributions' package managers, and greenfield C/C++ projects use the same model. I'd go so far as to say this model is a key reason why we need containerization to deploy modern software today - since you can't trust software written on distro X runs on distro Y because it packages dependency Z differently all the way up to the root of the dependency tree (glibc, usually). The fundamental flaw of this model is that it inverts the dependency tree. Build time dependencies are local to individual projects and need to be handled separately from all other software. With few exceptions, Linux distros make this mistake and it's why we can't rely on distro package management for new projects.
    - still_grokking 5y ago
      > What if the package you want isn't provided by the distribution? You package it. > What if the distribution's version doesn't support the features you need? You package the version that does. > What if it can't be built by relying only on things that are already packaged? You package the dependencies first. > How do you distribute your software so it runs on other distributions? Give them the source so they can package it. > Do you maintain a different build for each package manager? Yes, that's the whole idea behind distributions. > What if you want to run on platforms that don't have standard package managers, like MacOS or Windows? Well, there you anyway downloaded random things form the internet… But nowadays even those systems have package management that can be used! > How do you update the packages without internet downloads during provisioning the build? It's not about disabling package downloads (which come form a trusted source btw). It's about disabling downloads form random places on the internet. Also you can use a local package mirror. That's what the build systems of distributions do. > The C/C++ model has proven to be fatally flawed. It hasn't worked, that's why modern languages eschew reliance on the distributions' package managers, and greenfield C/C++ projects use the same model. Well, except for all packaged software out there… > I'd go so far as to say this model is a key reason why we need containerization to deploy modern software today Nobody needs that. Software form packages just works fine. All the Linux installations around the globe are a prove of that fact. > since you can't trust software written on distro X runs on distro Y because it packages dependency Z differently all the way up to the root of the dependency tree (glibc, usually). Of course you can. It works exceptionally fine. After you packaged it. > The fundamental flaw of this model is that it inverts the dependency tree. Build time dependencies are local to individual projects and need to be handled separately from all other software. With few exceptions, Linux distros make this mistake and it's why we can't rely on distro package management for new projects. More or less every distro does it wrong? And you know how to do it correctly? Maybe you should share your knowledge with the clueless people building distributions! Get in touch with for example Debian and tell them they need to stop making this mistake. By the way: How does your software reach the users when it's not packaged? At least I hear "sometimes" that users demand proper software packages. But maybe it's just me…

Yeah seems like basic hygiene especially given the supply chain attacks but also a lost cause. No one has the skills to do builds without the internet. Even forcing teams to use an allow list for the internet involving fighting a lot of angry people.

contingencies 5y ago
One of the few decent uses for containers is to enforce proxied internet so build process artifacts can be auto-stored for subsequent builds. For the worst offender I am aware of, try building a flutter project... it silent internet gets artifacts from at least three different packaging systems (node, cocoapods, android packages), all of which have caused hard-to-debug failures.

duped 5y ago

How do you build software in 2021 then? Do you write everything from scratch?

spoils19 5y ago
... yes? I've often been ridiculed for wanting to build everything in-house, but events like these just validate that sometimes, to get the most reliable and best software, you'll have to reinvent the wheel a few times.
- duped 5y ago
  To be honest that's deserving of ridicule. Reinventing wheels is how you introduce fragility and instability, not prevent it.
still_grokking 5y ago
One can use the packages provided by the distribution. That's how it worked since forever for things like C/C++. For example distris like Debian even mandate that you build software for them only relying on things that are already packaged. No arbitrary internet downloads allowed during build. The build environment contains only what is provided by the stated build-dependencies of your package, which are themself of course Debian packages, and some common minimal build base (like shell and some utils).
- duped 5y ago
  What if the package you want isn't provided by the distribution? What if the distribution's version doesn't support the features you need? What if it can't be built by relying only on things that are already packaged? How do you distribute your software so it runs on other distributions? Do you maintain a different build for each package manager? What if you want to run on platforms that don't have standard package managers, like MacOS or Windows? How do you update the packages without internet downloads during provisioning the build? The C/C++ model has proven to be fatally flawed. It hasn't worked, that's why modern languages eschew reliance on the distributions' package managers, and greenfield C/C++ projects use the same model. I'd go so far as to say this model is a key reason why we need containerization to deploy modern software today - since you can't trust software written on distro X runs on distro Y because it packages dependency Z differently all the way up to the root of the dependency tree (glibc, usually). The fundamental flaw of this model is that it inverts the dependency tree. Build time dependencies are local to individual projects and need to be handled separately from all other software. With few exceptions, Linux distros make this mistake and it's why we can't rely on distro package management for new projects.
  - still_grokking 5y ago
    > What if the package you want isn't provided by the distribution? You package it. > What if the distribution's version doesn't support the features you need? You package the version that does. > What if it can't be built by relying only on things that are already packaged? You package the dependencies first. > How do you distribute your software so it runs on other distributions? Give them the source so they can package it. > Do you maintain a different build for each package manager? Yes, that's the whole idea behind distributions. > What if you want to run on platforms that don't have standard package managers, like MacOS or Windows? Well, there you anyway downloaded random things form the internet… But nowadays even those systems have package management that can be used! > How do you update the packages without internet downloads during provisioning the build? It's not about disabling package downloads (which come form a trusted source btw). It's about disabling downloads form random places on the internet. Also you can use a local package mirror. That's what the build systems of distributions do. > The C/C++ model has proven to be fatally flawed. It hasn't worked, that's why modern languages eschew reliance on the distributions' package managers, and greenfield C/C++ projects use the same model. Well, except for all packaged software out there… > I'd go so far as to say this model is a key reason why we need containerization to deploy modern software today Nobody needs that. Software form packages just works fine. All the Linux installations around the globe are a prove of that fact. > since you can't trust software written on distro X runs on distro Y because it packages dependency Z differently all the way up to the root of the dependency tree (glibc, usually). Of course you can. It works exceptionally fine. After you packaged it. > The fundamental flaw of this model is that it inverts the dependency tree. Build time dependencies are local to individual projects and need to be handled separately from all other software. With few exceptions, Linux distros make this mistake and it's why we can't rely on distro package management for new projects. More or less every distro does it wrong? And you know how to do it correctly? Maybe you should share your knowledge with the clueless people building distributions! Get in touch with for example Debian and tell them they need to stop making this mistake. By the way: How does your software reach the users when it's not packaged? At least I hear "sometimes" that users demand proper software packages. But maybe it's just me…