6 ms·

From another angle: build systems. Currently building most libraries usually involves executing a series of highly complex byzantine scripts, requiring turing-

From another angle: build systems. Currently building most libraries usually involves executing a series of highly complex byzantine scripts, requiring turing-complete environment. This gives attacker an endless attack surface, and when the build process is hijacked - well, the opportunities are there. Moving to a declarative build process with only a limited state machine as an executor would help. Requiring all source blobs being reproducible might also be something to think about.

deleted 2y ago
[deleted]
kimixa 2y ago
Having a "simple" declarative build definition won't help if the thing that interprets and executes that definition is shipped with the package, as said interpreter is likely orders of magnitude more complex and harder to personally review. As is what happened with the xz example - the "attack" was hidden in code that is autogenerated by autotools, not the build or configuration definitions. People put trust in distros and packagers to having something of a review chain - there's 0% chance you personally be an expert in everything executing on your workstation right now (outside of maybe toy systems). I'm not an expert in m4 or bash, but I hope enough experts are in the chain to get to my distro's package library are that such things are less likely. But that is all bypassed here. I think this particular approach is a one-off, as I know of no other build environment where it's expected to have the generated executable of the build system "helpfully" packaged in the tarball as a packaging step. If it is in some I'm not aware of, I hope that decision is being re-examined now.
- AtlasBarfed 2y ago
  It needs money and people. Government funding for defense of the economy and computing in the "free world". Certainly the defense department has a billion to spare, and so does the EU
- MichaelZuo 2y ago
  This doesn't work for anything actually complex. For example if you've ever taken a look at the bluetooth specs you would not trust a single person in the world to implement it correctly and you probably wouldn't even trust an arbitrarily large team to implement it correctly. Unless they had a long demonstrated and credible track record of shipping perfectly functional products and an effectively unlimited budget, i. e. Apple and maybe 1 or 2 other groups, at most.
  - jancsika 2y ago
    > For example if you've ever taken a look at the bluetooth specs you would not trust a single person in the world to implement it correctly and you probably wouldn't even trust an arbitrarily large team to implement it correctly. I messed around a tiny bit with Bluetooth on Linux recently. Going to rankly speculate that Bluetooth is such a special case of hell such that it makes a distracting example here. I mean, as a joke suppose we wanted to design a 3.5 mm patch cord that pitch shifts down a 1/4 step for randomly chosen stretches. It turns out to be easy-- just remove the wire from the casing and replace it with cheap bluetooth chips at either end. You'll get that behavior for free! :) Compare to, say, USB, where your point above would apply just as well. I wouldn't be distracted by that example because even cheapo, decades-old USB drives to this day let me read/write without interleaving zeros in my data. Shit, now I'm distracted again. Does Bluetooth audio even have the notion of a buffer size that I can set from the sender? And I guess technically the receiver isn't interleaving the signal with zeros-- it's adjusting the rate at which it sends blocks of the received data to the audio subsystem. Was Bluetooth audio basically designed just for the human voice under the assumption we're constantly pitch shifting? Oops, I almost forgot-- the sample-rate shift is preceded by a dropout, so I do get interleaved zeros in the audio data! I actually get a smoother ramp in my Prius changing from battery to ICE than I do in my wireless audio system!!! Anyhow, what were we talking about again?
- bonzini 2y ago
  No, the attack was not hidden in code that is autogenerated by autotools. If that was the case, rerunning autoconf (which most distros do) would have disabled the attack. Instead it was hidden in plain sight in the source that is processed by autoconf. > it's expected to have the generated executable of the build system "helpfully" packaged in the tarball as a packaging step. While this is true, most distros as I mentioned above rerun autoconf to ensure consistency in the tests and possibly to include autoconf bug fixes.
  - rcxdude 2y ago
    The jumping off point was modified autotools output. re-running autogen did effectively disable the attack. The payload was stashed in some test files, but the build system needed to jump through quite some hoops to actually get that into the compiled library. Apparently the target distributions didn't re-run autogen for this package.
    - masfuerte 2y ago
      This is what the early reporting said but the article has additional info. The code to include the backdoor in the build was in an m4 script. The initial reporting said that this code was not present in the github source, but the post-autogen code (including the attack) was included in the github releases. The article says that this modified script was present in the source on tukaani.org, which was controlled by the attacker and used by the distros as their upstream. If you downloaded from github and reran autogen you were OK. If you downloaded from tukaani and reran autogen, like the distros did, you lost.
    - deleted 2y ago
      [deleted]
    - markhahn 2y ago
      merely keeping build and test environments isolated would have entirely prevented this attack. better hygeine that way would also simplify both environments, since the tools for each are fairly different.
nonrandomstring 2y ago
Yes. Other than the vulnerability of developers/maintainers the other big takeaway I get from this incident is that build systems have become dangerously unwieldy. There's just too many moving parts and too many places to hide bad stuff.
- tsimionescu 2y ago
  Build systems have always been like this. It is in fact more recent build systems that are limiting the amount of crazyness you can do compared to the older ones.
duped 2y ago
This is how we got autoconf in the first place - an under specified and under powered build configuration language (make) which led to people that did need arbitrary logic to generate the build files at configuration time. Don't limit the build system language. It makes builds more complex, not less.

deleted 2y ago

[deleted]

kimixa 2y ago

Having a "simple" declarative build definition won't help if the thing that interprets and executes that definition is shipped with the package, as said interpreter is likely orders of magnitude more complex and harder to personally review. As is what happened with the xz example - the "attack" was hidden in code that is autogenerated by autotools, not the build or configuration definitions. People put trust in distros and packagers to having something of a review chain - there's 0% chance you personally be an expert in everything executing on your workstation right now (outside of maybe toy systems). I'm not an expert in m4 or bash, but I hope enough experts are in the chain to get to my distro's package library are that such things are less likely. But that is all bypassed here. I think this particular approach is a one-off, as I know of no other build environment where it's expected to have the generated executable of the build system "helpfully" packaged in the tarball as a packaging step. If it is in some I'm not aware of, I hope that decision is being re-examined now.

AtlasBarfed 2y ago
It needs money and people. Government funding for defense of the economy and computing in the "free world". Certainly the defense department has a billion to spare, and so does the EU
MichaelZuo 2y ago
This doesn't work for anything actually complex. For example if you've ever taken a look at the bluetooth specs you would not trust a single person in the world to implement it correctly and you probably wouldn't even trust an arbitrarily large team to implement it correctly. Unless they had a long demonstrated and credible track record of shipping perfectly functional products and an effectively unlimited budget, i. e. Apple and maybe 1 or 2 other groups, at most.
- jancsika 2y ago
  > For example if you've ever taken a look at the bluetooth specs you would not trust a single person in the world to implement it correctly and you probably wouldn't even trust an arbitrarily large team to implement it correctly. I messed around a tiny bit with Bluetooth on Linux recently. Going to rankly speculate that Bluetooth is such a special case of hell such that it makes a distracting example here. I mean, as a joke suppose we wanted to design a 3.5 mm patch cord that pitch shifts down a 1/4 step for randomly chosen stretches. It turns out to be easy-- just remove the wire from the casing and replace it with cheap bluetooth chips at either end. You'll get that behavior for free! :) Compare to, say, USB, where your point above would apply just as well. I wouldn't be distracted by that example because even cheapo, decades-old USB drives to this day let me read/write without interleaving zeros in my data. Shit, now I'm distracted again. Does Bluetooth audio even have the notion of a buffer size that I can set from the sender? And I guess technically the receiver isn't interleaving the signal with zeros-- it's adjusting the rate at which it sends blocks of the received data to the audio subsystem. Was Bluetooth audio basically designed just for the human voice under the assumption we're constantly pitch shifting? Oops, I almost forgot-- the sample-rate shift is preceded by a dropout, so I do get interleaved zeros in the audio data! I actually get a smoother ramp in my Prius changing from battery to ICE than I do in my wireless audio system!!! Anyhow, what were we talking about again?
bonzini 2y ago
No, the attack was not hidden in code that is autogenerated by autotools. If that was the case, rerunning autoconf (which most distros do) would have disabled the attack. Instead it was hidden in plain sight in the source that is processed by autoconf. > it's expected to have the generated executable of the build system "helpfully" packaged in the tarball as a packaging step. While this is true, most distros as I mentioned above rerun autoconf to ensure consistency in the tests and possibly to include autoconf bug fixes.
- rcxdude 2y ago
  The jumping off point was modified autotools output. re-running autogen did effectively disable the attack. The payload was stashed in some test files, but the build system needed to jump through quite some hoops to actually get that into the compiled library. Apparently the target distributions didn't re-run autogen for this package.
  - masfuerte 2y ago
    This is what the early reporting said but the article has additional info. The code to include the backdoor in the build was in an m4 script. The initial reporting said that this code was not present in the github source, but the post-autogen code (including the attack) was included in the github releases. The article says that this modified script was present in the source on tukaani.org, which was controlled by the attacker and used by the distros as their upstream. If you downloaded from github and reran autogen you were OK. If you downloaded from tukaani and reran autogen, like the distros did, you lost.
  - deleted 2y ago
    [deleted]
  - markhahn 2y ago
    merely keeping build and test environments isolated would have entirely prevented this attack. better hygeine that way would also simplify both environments, since the tools for each are fairly different.

nonrandomstring 2y ago

Yes. Other than the vulnerability of developers/maintainers the other big takeaway I get from this incident is that build systems have become dangerously unwieldy. There's just too many moving parts and too many places to hide bad stuff.

tsimionescu 2y ago
Build systems have always been like this. It is in fact more recent build systems that are limiting the amount of crazyness you can do compared to the older ones.

duped 2y ago

This is how we got autoconf in the first place - an under specified and under powered build configuration language (make) which led to people that did need arbitrary logic to generate the build files at configuration time. Don't limit the build system language. It makes builds more complex, not less.