3 ms·

This is my main take-away from this. We must stop using upstream configure and other "binary" scripts. Delete them all and run "autoreconf -fi" to recreate th

by rwmj 3y ago

This is my main take-away from this. We must stop using upstream configure and other "binary" scripts. Delete them all and run "autoreconf -fi" to recreate them. (Debian already does something like this I think.)

anthk 3y ago
I always run autoreconfig -ifv first.
- rwmj 3y ago
  In this case it wouldn't be sufficient. You had to also delete m4/build-to-host.m4 for autoreconf to recreate it.
  - anthk 3y ago
    Thanks. At least the distro I use (Hyperbola) it's LTS bound, so is not affected.
cesarb 3y ago
> We must stop using upstream configure and other "binary" scripts. Delete them all and run "autoreconf -fi" to recreate them. I would go further than that: all files which are in a distributed tarball, but not on the corresponding git repository, should be treated as suspect. Distributing these generated autotools files is a relic of times when it could not be expected that the target machine would have all the necessary development environment pieces. Nowadays, we should be able to assume that whoever wants to compile the code can also run autoconf/automake/etc to generate the build scripts from their sources. And other than the autotools output, and perhaps a couple of other tarball build artifacts (like cargo simplifying the Cargo.toml file), there should be no difference between what is distributed and what is on the repository. I recall reading about some project to find the corresponding commit for all Rust crates and compare it with the published crate, though I can't find it right now; I don't know whether there's something similar being done for other ecosystems.
- nolist_policy 3y ago
  One small problem with this is that autoconf is not backwards-compatible. There are projects out there that need older autoconf than distributions ship with.
  - rwmj 3y ago
    There are, and they need to be fixed.
  - cesarb 3y ago
    The test code generated by older autoconf is not going to be work correctly with newer GCC releases due to the deprecation of implicit int and implicit function declarations (see https://fedoraproject.org/wiki/Changes/PortingToModernC https://fedoraproject.org/wiki/Changes/PortingToModernC), so these projects already have to be updated to work with newer autoconf.
    - badsectoracula 3y ago
      Typing `./configure` wont work but something like `./configure CFLAGS="-Wno-error=implicit-function-declaration"` (or whatever flag) might work (IIRC it is possible to pass flags to the compiler invocations used for checking out the existence of features) without needing to recreate it. Also chances are you can shove that flag in some old `configure.in` and have it work with an old autoconf for years before it having to update it :-P.
  - Too 3y ago
    Easily solved with Docker. Yes, it sucks to add yet another wrapper but that’s what you get for choosing non backwards compatible tools in the first place. In combination with projects that don’t keep up to date on supporting later versions.
    - account42 3y ago
      A yes, let's replace "binary" blobs in release archives with even bigger blobs of mistery goo.
nicolas_17 3y ago
The backdoor is in an .m4 file that gets parsed by autoconf to generate the configure script. Running autoconf yourself won't save you.
- rwmj 3y ago
  That's not entirely true. autoreconf will regenerate m4/build-to-host.m4 but only if you delete it first.
tobias2014 3y ago
It seems like this was the solution for archlinux, pull directly from the github tag and run autogen: https://gitlab.archlinux.org/archlinux/packaging/packages/xz/-/commit/881385757abdc39d3cfea1c3e34ec09f637424ad https://gitlab.archlinux.org/archlinux/packaging/packages/xz...
- 1oooqooq 3y ago
  it's shocking how many packages on distros are just one random tarball from the internet with lipstick
k8svet 3y ago
Oh come on, please, let's put autotools out to pasture. I've lost so much of my life fighting autotools crap compared to "just use meson".
- account42 3y ago
  As long as we also exterminate it's even more evil brother - libtool.
snnn 3y ago
I don't think it would help much. I work on machine learning frameworks. A lot of them(and math libraries) rely on just in time compilation. None of us has the time or expertise to inspect JIT-ed assembly code. Not even mentioning that much of the code deliberately read/write out of bound, which is not an issue if you always add some extra bytes at the end of each buffer, which could make most memory sanitizer tools useless. When you run their unit tests, you run the JIT code, then a lot of things could happen. Maybe we should ask all packaging systems splitting their build into compile and test two stages, to ensure that a testing code would not impact the binaries that are going to be published. I would rather to read and analysis the generated code instead of the code that generates it.

by rwmj 3y ago

anthk 3y ago

I always run autoreconfig -ifv first.

rwmj 3y ago
In this case it wouldn't be sufficient. You had to also delete m4/build-to-host.m4 for autoreconf to recreate it.
- anthk 3y ago
  Thanks. At least the distro I use (Hyperbola) it's LTS bound, so is not affected.

cesarb 3y ago

> We must stop using upstream configure and other "binary" scripts. Delete them all and run "autoreconf -fi" to recreate them. I would go further than that: all files which are in a distributed tarball, but not on the corresponding git repository, should be treated as suspect. Distributing these generated autotools files is a relic of times when it could not be expected that the target machine would have all the necessary development environment pieces. Nowadays, we should be able to assume that whoever wants to compile the code can also run autoconf/automake/etc to generate the build scripts from their sources. And other than the autotools output, and perhaps a couple of other tarball build artifacts (like cargo simplifying the Cargo.toml file), there should be no difference between what is distributed and what is on the repository. I recall reading about some project to find the corresponding commit for all Rust crates and compare it with the published crate, though I can't find it right now; I don't know whether there's something similar being done for other ecosystems.

nolist_policy 3y ago
One small problem with this is that autoconf is not backwards-compatible. There are projects out there that need older autoconf than distributions ship with.
- rwmj 3y ago
  There are, and they need to be fixed.
- cesarb 3y ago
  The test code generated by older autoconf is not going to be work correctly with newer GCC releases due to the deprecation of implicit int and implicit function declarations (see https://fedoraproject.org/wiki/Changes/PortingToModernC https://fedoraproject.org/wiki/Changes/PortingToModernC), so these projects already have to be updated to work with newer autoconf.
  - badsectoracula 3y ago
    Typing `./configure` wont work but something like `./configure CFLAGS="-Wno-error=implicit-function-declaration"` (or whatever flag) might work (IIRC it is possible to pass flags to the compiler invocations used for checking out the existence of features) without needing to recreate it. Also chances are you can shove that flag in some old `configure.in` and have it work with an old autoconf for years before it having to update it :-P.
- Too 3y ago
  Easily solved with Docker. Yes, it sucks to add yet another wrapper but that’s what you get for choosing non backwards compatible tools in the first place. In combination with projects that don’t keep up to date on supporting later versions.
  - account42 3y ago
    A yes, let's replace "binary" blobs in release archives with even bigger blobs of mistery goo.

nicolas_17 3y ago

The backdoor is in an .m4 file that gets parsed by autoconf to generate the configure script. Running autoconf yourself won't save you.

rwmj 3y ago
That's not entirely true. autoreconf will regenerate m4/build-to-host.m4 but only if you delete it first.

tobias2014 3y ago

It seems like this was the solution for archlinux, pull directly from the github tag and run autogen: https://gitlab.archlinux.org/archlinux/packaging/packages/xz/-/commit/881385757abdc39d3cfea1c3e34ec09f637424ad https://gitlab.archlinux.org/archlinux/packaging/packages/xz...

1oooqooq 3y ago
it's shocking how many packages on distros are just one random tarball from the internet with lipstick

k8svet 3y ago

Oh come on, please, let's put autotools out to pasture. I've lost so much of my life fighting autotools crap compared to "just use meson".

account42 3y ago
As long as we also exterminate it's even more evil brother - libtool.

snnn 3y ago

I don't think it would help much. I work on machine learning frameworks. A lot of them(and math libraries) rely on just in time compilation. None of us has the time or expertise to inspect JIT-ed assembly code. Not even mentioning that much of the code deliberately read/write out of bound, which is not an issue if you always add some extra bytes at the end of each buffer, which could make most memory sanitizer tools useless. When you run their unit tests, you run the JIT code, then a lot of things could happen. Maybe we should ask all packaging systems splitting their build into compile and test two stages, to ensure that a testing code would not impact the binaries that are going to be published. I would rather to read and analysis the generated code instead of the code that generates it.