3 ms·

> We must stop using upstream configure and other "binary" scripts. Delete them all and run "autoreconf -fi" to recreate them. I would go further than that: al

by cesarb 3y ago

> We must stop using upstream configure and other "binary" scripts. Delete them all and run "autoreconf -fi" to recreate them. I would go further than that: all files which are in a distributed tarball, but not on the corresponding git repository, should be treated as suspect. Distributing these generated autotools files is a relic of times when it could not be expected that the target machine would have all the necessary development environment pieces. Nowadays, we should be able to assume that whoever wants to compile the code can also run autoconf/automake/etc to generate the build scripts from their sources. And other than the autotools output, and perhaps a couple of other tarball build artifacts (like cargo simplifying the Cargo.toml file), there should be no difference between what is distributed and what is on the repository. I recall reading about some project to find the corresponding commit for all Rust crates and compare it with the published crate, though I can't find it right now; I don't know whether there's something similar being done for other ecosystems.

nolist_policy 3y ago
One small problem with this is that autoconf is not backwards-compatible. There are projects out there that need older autoconf than distributions ship with.
- rwmj 3y ago
  There are, and they need to be fixed.
- cesarb 3y ago
  The test code generated by older autoconf is not going to be work correctly with newer GCC releases due to the deprecation of implicit int and implicit function declarations (see https://fedoraproject.org/wiki/Changes/PortingToModernC https://fedoraproject.org/wiki/Changes/PortingToModernC), so these projects already have to be updated to work with newer autoconf.
  - badsectoracula 3y ago
    Typing `./configure` wont work but something like `./configure CFLAGS="-Wno-error=implicit-function-declaration"` (or whatever flag) might work (IIRC it is possible to pass flags to the compiler invocations used for checking out the existence of features) without needing to recreate it. Also chances are you can shove that flag in some old `configure.in` and have it work with an old autoconf for years before it having to update it :-P.
- Too 3y ago
  Easily solved with Docker. Yes, it sucks to add yet another wrapper but that’s what you get for choosing non backwards compatible tools in the first place. In combination with projects that don’t keep up to date on supporting later versions.
  - account42 3y ago
    A yes, let's replace "binary" blobs in release archives with even bigger blobs of mistery goo.
- pajko 3y ago
  https://www.gnu.org/software/autoconf-archive/ https://www.gnu.org/software/autoconf-archive/
londons_explore 3y ago
Why do we distribute tarballs at all? A git hash should be all thats needed...
- cesarb 3y ago
  > Why do we distribute tarballs at all? A git hash should be all thats needed... A git hash means nothing without the repository it came from, so you'd need to distribute both. A tarball is a self-contained artifact. If I store a tarball in a CD-ROM, and look at it twenty years later, it will still have the same complete code; if I store a git hash in a CD-ROM, without storing a copy of the repository together with it, twenty years later there's a good chance that the repository is no longer available. We could distribute the git hash together with a shallow copy of the repository (we don't actually need the history as long as the commit with its trees and blobs is there), but that's just reinventing a tarball with more steps. (Setting aside that currently git hashes use SHA-1, which is not considered strong enough.)
  - londons_explore 3y ago
    except it isn't reinventing the tarball, because the git hash forces verification that every single file in the repo matches that in the release. And git even has support for "compressed git repo in a file" or "shallow git repo in a file" or even "diffs from the last release, compressed in a file". They're called "git bundle"'s. They're literally perfect for software distribution and archiving.
    - bombcar 3y ago
      People don't know how to use git hashes, and it's not been "done". Whereas downloading tarballs and verifying hashes of the tarball has been "good enough" because the real thing it's been detecting is communication faults, not supply chain attacks. People also like version numbers like 2.5.1 but that's not a hash, and you can only indirectly make it a hash.
hypnagogic 3y ago
> I would go further than that: all files which are in a distributed tarball, but not on the corresponding git repository, should be treated as suspect. This and the automated A/B / diff to check the tarball against the repo, flag if mismatched.

by cesarb 3y ago

nolist_policy 3y ago

One small problem with this is that autoconf is not backwards-compatible. There are projects out there that need older autoconf than distributions ship with.

rwmj 3y ago
There are, and they need to be fixed.
cesarb 3y ago
The test code generated by older autoconf is not going to be work correctly with newer GCC releases due to the deprecation of implicit int and implicit function declarations (see https://fedoraproject.org/wiki/Changes/PortingToModernC https://fedoraproject.org/wiki/Changes/PortingToModernC), so these projects already have to be updated to work with newer autoconf.
- badsectoracula 3y ago
  Typing `./configure` wont work but something like `./configure CFLAGS="-Wno-error=implicit-function-declaration"` (or whatever flag) might work (IIRC it is possible to pass flags to the compiler invocations used for checking out the existence of features) without needing to recreate it. Also chances are you can shove that flag in some old `configure.in` and have it work with an old autoconf for years before it having to update it :-P.
Too 3y ago
Easily solved with Docker. Yes, it sucks to add yet another wrapper but that’s what you get for choosing non backwards compatible tools in the first place. In combination with projects that don’t keep up to date on supporting later versions.
- account42 3y ago
  A yes, let's replace "binary" blobs in release archives with even bigger blobs of mistery goo.
pajko 3y ago
https://www.gnu.org/software/autoconf-archive/ https://www.gnu.org/software/autoconf-archive/

londons_explore 3y ago

Why do we distribute tarballs at all? A git hash should be all thats needed...

cesarb 3y ago
> Why do we distribute tarballs at all? A git hash should be all thats needed... A git hash means nothing without the repository it came from, so you'd need to distribute both. A tarball is a self-contained artifact. If I store a tarball in a CD-ROM, and look at it twenty years later, it will still have the same complete code; if I store a git hash in a CD-ROM, without storing a copy of the repository together with it, twenty years later there's a good chance that the repository is no longer available. We could distribute the git hash together with a shallow copy of the repository (we don't actually need the history as long as the commit with its trees and blobs is there), but that's just reinventing a tarball with more steps. (Setting aside that currently git hashes use SHA-1, which is not considered strong enough.)
- londons_explore 3y ago
  except it isn't reinventing the tarball, because the git hash forces verification that every single file in the repo matches that in the release. And git even has support for "compressed git repo in a file" or "shallow git repo in a file" or even "diffs from the last release, compressed in a file". They're called "git bundle"'s. They're literally perfect for software distribution and archiving.
  - bombcar 3y ago
    People don't know how to use git hashes, and it's not been "done". Whereas downloading tarballs and verifying hashes of the tarball has been "good enough" because the real thing it's been detecting is communication faults, not supply chain attacks. People also like version numbers like 2.5.1 but that's not a hash, and you can only indirectly make it a hash.

hypnagogic 3y ago

> I would go further than that: all files which are in a distributed tarball, but not on the corresponding git repository, should be treated as suspect. This and the automated A/B / diff to check the tarball against the repo, flag if mismatched.