3 ms·

This is not going to be popular: I think the whole idea that a build system just fetches resources from outside of the build environment is fundamentally broken

This is not going to be popular: I think the whole idea that a build system just fetches resources from outside of the build environment is fundamentally broken. It invites all kinds of trouble and makes it next to impossible to really achieve stability and to ensure that all code that is in the build has been verified. Because after you've done it four times the fifth time you won't be looking closely. But if you don't do it automatically but only when you actually need it you will be looking a lot more sharpish at what has changed since you last pulled in the code. Especially for older and stable libraries the consumers should dictate when they upgrade, not some automatic build process. But because we're all conditioned to download stuff because it may have solved some security issue we stopped to think about the security issues associated with just downloading stuff and dumping it into the build process.

kragen 1y ago
I am pretty sure Debian Policy agrees with you, although I can't cite chapter and verse. Certainly Nix and Guix agree with you. But that evidently wasn't the problem here.
Sophira 1y ago
I completely agree with you - I think that automatic downloading of dependencies when building is a bad idea. However, for the sake of devil's advocacy, I do also want to point out that the first thing a lot of people used to do after downloading and extracting a source tarball was to run "./configure" without even looking at what it is they were executing - even people who (rightly) hate the "curl | bash" combo. You could be running anything. Being able to verify what it is you're running is vitally important, but in the end it only makes a difference if people take the time to do so. (And running "./configure --help" doesn't count.)
- imoverclocked 1y ago
  Adding sudo in front makes it secure though, right? /sarcasm -- Automatic downloading of dependencies can be done in a sane way but not sane without significant effort. eg: building Debian packages can install other pre-packaged dependencies. In theory other packages are built the same way. Where this becomes an issue specifically is where language-specific mechanisms reach-out and just install dependencies. To be fair, this has happened for a long time (I'm looking at you, CPAN) and does provide a lot of potential utility to any given developer. What might be better than advocating for "not doing this at all" is "fixing the pattern." There are probably better ideas than this but I'll start here: 1) make repositories include different levels of software by default. core vs community at an absolute minimum. Maybe add some levels like, "vetted versions" or "gist" to convey information about quality or security. 2) make it easy to have locally vetted pools to choose from. eg: Artifactory makes it easy to locally cache upstream software repos which is a good start. Making a vetting process out of that would be ... useful but cumbersome. At the end of the day, we are always running someone else's code.
- bmandale 1y ago
  > even people who (rightly) hate the "curl | bash" combo. You could be running anything. That's true unless I audit every single line, out of potentially millions, in the source of a program I intend to run. If I'm going to do that, then I could audit the ./configure script as well.
  - uecker 1y ago
    This is missing the point. The issue with "configure" is that it is easy to hide malicious code in it because it is so arcane. The issue with "curl | bash" is that - on top - there is not even a proper trust chain and independent verification. "curl | bash" needs to die. Any project who promotes this does not care or does not understand security. "configure" was a necessary evil in the past with all commercial UNIX working differently. Nowadays I think it should go away.
    - jacquesm 1y ago
      Autotools is a hot mess. Anything complex is going to be a rich environment for exploits of all kinds. The more silent the exploit the bigger the chance that it will spread widely.
    - dns_snek 1y ago
      > "curl | bash" needs to die. Any project who promotes this does not care or does not understand security. Do you "understand security"? There's a grain of truth to what you're saying, but not more than that. The crux of this problem is with running untrusted binaries (or unreviewed source code) vs. installing something from a trusted repository. The majority of people either don't know or don't care to review the source code. They simply run the commands displayed on the website, and whether you ask them to "curl | bash" or "wget && apt install ./some.deb" won't make any difference to their security. Even if you do a "proper trust chain" and digitally sign your packages, that key is served through the same channel as the installation instructions and thus requires trust on first use, just like "curl | bash". Unfortunately publishing every piece of software through every operating system's default repository isn't very realistic. Someone, somewhere is going to have to install the binary manually and "curl | bash" is as good of a method for doing that as any.
- nmz 1y ago
  I would like to add that sudo make install is a bigger security risk and there is absolutely no need to run make install as root when you could target a directory that mimics / and tar it with the appropriate root permissions leaving only the extraction as root, you could even take a snapshot of the system and undo on error. All done via coreutils.
- frizlab 1y ago
  > I think that automatic downloading of dependencies when building is a bad idea. Unless the dependencies are properly pinned and hashed.
dataflow 1y ago
> I think the whole idea that a build system just fetches resources from outside of the build environment is fundamentally broken I think your phrasing is a bit overbroad. There's nothing fundamentally broken with the build system fetching resources; what's broken is not verifying what it's fetching. Audit the package beforehand and have your build system verify its integrity after downloading, and you're fine.
- jacquesm 1y ago
  xz. nobody verifies all packages that are automatically downloaded all the time, unless there is a problem. We got lucky, that time.
gizmo686 1y ago
The solution I've seen employed is to prevent the build environment from reaching outside. Setup a mirror of all the repositories you care about; then configure the network so your build system can reach the mirrors; but not the general Internet. Of course, once you do this, you eventually create a cron job on mirrors to blindly update themselves... This setup does at least prevent an old version of a dependency from silently changing, so projects that pin their dependencies can be confident in that. But even in those cases, you end up with a periodic "update all dependencies" ticket, that just blindly takes the new version.

kragen 1y ago

I am pretty sure Debian Policy agrees with you, although I can't cite chapter and verse. Certainly Nix and Guix agree with you. But that evidently wasn't the problem here.

Sophira 1y ago

I completely agree with you - I think that automatic downloading of dependencies when building is a bad idea. However, for the sake of devil's advocacy, I do also want to point out that the first thing a lot of people used to do after downloading and extracting a source tarball was to run "./configure" without even looking at what it is they were executing - even people who (rightly) hate the "curl | bash" combo. You could be running anything. Being able to verify what it is you're running is vitally important, but in the end it only makes a difference if people take the time to do so. (And running "./configure --help" doesn't count.)

imoverclocked 1y ago
Adding sudo in front makes it secure though, right? /sarcasm -- Automatic downloading of dependencies can be done in a sane way but not sane without significant effort. eg: building Debian packages can install other pre-packaged dependencies. In theory other packages are built the same way. Where this becomes an issue specifically is where language-specific mechanisms reach-out and just install dependencies. To be fair, this has happened for a long time (I'm looking at you, CPAN) and does provide a lot of potential utility to any given developer. What might be better than advocating for "not doing this at all" is "fixing the pattern." There are probably better ideas than this but I'll start here: 1) make repositories include different levels of software by default. core vs community at an absolute minimum. Maybe add some levels like, "vetted versions" or "gist" to convey information about quality or security. 2) make it easy to have locally vetted pools to choose from. eg: Artifactory makes it easy to locally cache upstream software repos which is a good start. Making a vetting process out of that would be ... useful but cumbersome. At the end of the day, we are always running someone else's code.
bmandale 1y ago
> even people who (rightly) hate the "curl | bash" combo. You could be running anything. That's true unless I audit every single line, out of potentially millions, in the source of a program I intend to run. If I'm going to do that, then I could audit the ./configure script as well.
- uecker 1y ago
  This is missing the point. The issue with "configure" is that it is easy to hide malicious code in it because it is so arcane. The issue with "curl | bash" is that - on top - there is not even a proper trust chain and independent verification. "curl | bash" needs to die. Any project who promotes this does not care or does not understand security. "configure" was a necessary evil in the past with all commercial UNIX working differently. Nowadays I think it should go away.
  - jacquesm 1y ago
    Autotools is a hot mess. Anything complex is going to be a rich environment for exploits of all kinds. The more silent the exploit the bigger the chance that it will spread widely.
  - dns_snek 1y ago
    > "curl | bash" needs to die. Any project who promotes this does not care or does not understand security. Do you "understand security"? There's a grain of truth to what you're saying, but not more than that. The crux of this problem is with running untrusted binaries (or unreviewed source code) vs. installing something from a trusted repository. The majority of people either don't know or don't care to review the source code. They simply run the commands displayed on the website, and whether you ask them to "curl | bash" or "wget && apt install ./some.deb" won't make any difference to their security. Even if you do a "proper trust chain" and digitally sign your packages, that key is served through the same channel as the installation instructions and thus requires trust on first use, just like "curl | bash". Unfortunately publishing every piece of software through every operating system's default repository isn't very realistic. Someone, somewhere is going to have to install the binary manually and "curl | bash" is as good of a method for doing that as any.
nmz 1y ago
I would like to add that sudo make install is a bigger security risk and there is absolutely no need to run make install as root when you could target a directory that mimics / and tar it with the appropriate root permissions leaving only the extraction as root, you could even take a snapshot of the system and undo on error. All done via coreutils.
frizlab 1y ago
> I think that automatic downloading of dependencies when building is a bad idea. Unless the dependencies are properly pinned and hashed.

dataflow 1y ago

> I think the whole idea that a build system just fetches resources from outside of the build environment is fundamentally broken I think your phrasing is a bit overbroad. There's nothing fundamentally broken with the build system fetching resources; what's broken is not verifying what it's fetching. Audit the package beforehand and have your build system verify its integrity after downloading, and you're fine.

jacquesm 1y ago
xz. nobody verifies all packages that are automatically downloaded all the time, unless there is a problem. We got lucky, that time.

gizmo686 1y ago

The solution I've seen employed is to prevent the build environment from reaching outside. Setup a mirror of all the repositories you care about; then configure the network so your build system can reach the mirrors; but not the general Internet. Of course, once you do this, you eventually create a cron job on mirrors to blindly update themselves... This setup does at least prevent an old version of a dependency from silently changing, so projects that pin their dependencies can be confident in that. But even in those cases, you end up with a periodic "update all dependencies" ticket, that just blindly takes the new version.