10 ms·

From xz to ibus: more questionable tarballs

170 points by nateb2022 3y ago

dhx 3y ago
1.5.29-rc2 was tagged on 9 Nov 2023 [1] and, as an example, did not contain "N_("CJK Unified Ideographs Extension I")," in src/ibusunicodegen.h [2]. Commit 228f0a77b2047ade54e132bab69c0c03f0f41aae from 28 Feb 2023 introduced this change instead. It's the same person who tagged 1.5.29-rc2 and committed 228f0a77b2047ade54e132bab69c0c03f0f41aae which is typically an indication the maintainer tar'd their checked out git folder and accidentally included changes not get committed. The question raised is whether anyone is auditing these differences before the checksummed tarballs are added to package repositories. [1] https://github.com/ibus/ibus/releases/tag/1.5.29-rc2 https://github.com/ibus/ibus/releases/tag/1.5.29-rc2 [2] https://github.com/ibus/ibus/blob/0ad8e77bd36545974ad8acd0a5283cf72bc7c8ad/src/ibusunicodegen.h https://github.com/ibus/ibus/blob/0ad8e77bd36545974ad8acd0a5... [3] https://github.com/ibus/ibus/commit/228f0a77b2047ade54e132bab69c0c03f0f41aae https://github.com/ibus/ibus/commit/228f0a77b2047ade54e132ba...
- dhx 3y ago
  ^ typo: 28 Feb 2023 is meant to be 28 Feb 2024, or almost 4 months later.
- juitpykyk 3y ago
  GitHub has the feature of downloading the tree as a zip, why is this not used?
  - dhx 3y ago
    For many projects, the release tarballs only contain the files necessary to build the software, and not the following items that may be present in the same repository: - scripts used by project developers to import translations from another system and commit those translation changes to the repository - scripts to build release tarballs and sign them - continuous integration scripts - configuration and scripts used to setup a developer IDE environment
    - juitpykyk 3y ago
      What would be the problem of downloading a few MB more, aren't these source tarballs just used to build the distro binary and then they are deleted?
      - dhx 3y ago
        As this backdoor has shown, extra unnecessary files in the source files can make it easier to hide malicious code. If you take Gentoo as an example, when a software package is built, Gentoo creates a sandboxed environment first, disallowing the build process from impacting the rest of the operating system.[1] Removing superfluous files from the source tarballs minimises the ability for an attacker to get malicious code inside the sandboxed build environment. Sandboxes for building software are commonly used throughout Linux distributions, but I am unsure how strict those sandboxes are in general e.g. whether they use seccomp and really tighten what a build script can get up to. At least on Gentoo, there is a subset of packages (such as GNU coreutils) that are always just assumed to be needed to build software and they're always present in the sandbox. Build dependencies aren't as granular as "this build needs to use awk but not sed". [1] https://wiki.gentoo.org/wiki/Sandbox_(Portage) https://wiki.gentoo.org/wiki/Sandbox_(Portage)
    - roywashere 3y ago
      You can use .gitattributes export-ignore to influence what gets into the tarballs and what stays into the repository! It's super powerful but not often used
      - kzrdude 3y ago
        And export-subst to insert the current tag or git revision into the archive too. In fact export-subst is powerful enough that there is probably some way to create an exploit triggered by a particular payload inside a commit or tag message? :) Maybe not triggered, but it could be part of the chain.
      - bdd8f1df777b 3y ago
        I smell a new backdooring opportunity. Modifying .gitattributes to surreptitiously sneak some binary files into the GitHub release tarballs. Few poeple would take a look at .gitattributes.
  - rwmj 3y ago
    There are two main reasons, one bad, one good(-ish): (1) Traditional autoconf assumed you only had a shell and cc and make, and so the ./configure script which is a huge ball of obscure shell commands is shipped. I think most distros will now delete and recreate these files, which probably should have happened a lot earlier. (Debian has been mostly doing this right for a long time already.) (2) Some programs generate a lot of code (eg. in libnbd we generate thousands of lines of boilerplate C from API descriptions*). To avoid people needing to install the specific tools that we use to generate that code, we distribute the generated files in the tarball, but it's not present in git. You can still build from git directly, and you can also verify the generated code exactly matches the tarball, but both cases mean extra build dependencies for end users and packagers. * Generating boilerplate code is a good thing in general as it reduces systematic errors, which are a vastly more common source of bugs compared to highly targeted supply chain attacks.
    - efrecon 3y ago
      Downloading the tarball/zip with just a shell and regular utils is possible. See https://github.com/efrecon/ungit https://github.com/efrecon/ungit
    - watt 3y ago
      I advocate for checking in the auto-generated code. You can see the differences between the tool runs, can see how changes in tooling affect the generated code, can see what might have caused a regression (hey it happens). Sometimes tooling can generate unstable files, I recall there was time when Eclipse was notorious there, for example when saving XML files they liked to reorder all the attributes. But these are bugs that need to be fixed. Tooling should generate perfectly reproducible files.
      - rwmj 3y ago
        We started off doing this, but you end up with enormous diffs which are themselves confusing. Example, only about 5% of this change is non-generated: https://github.com/libguestfs/libguestfs/commit/5186251f8f681f2ebb028423bb49a748861fd11e https://github.com/libguestfs/libguestfs/commit/5186251f8f68... Probably depends on the project as to whether this is feasible, but for us we intentionally want to generate everything we can in order to reduce systematic errors.
  - Lockal 3y ago
    There is a minor note that technically there is a weak guarantee that checksums won't break after server update and recompression with different version / alternative implementation of gzip. https://github.com/orgs/community/discussions/45830 https://github.com/orgs/community/discussions/45830
    - Aissen 3y ago
      It's not a minor note, it's a major reason that the github auto-generated tarballs are useless as-is, since they are not stable.
      - rwmj 3y ago
        github have (for the moment) backed down and currently the auto-generated tarballs are stable, but they have in the past and may in the future change this.
      - colejohnson66 3y ago
        This was not GitHub’s fault, but Git itself combined with cache pruning. Specifically, GitHub updating to Git 2.38 which changed the algorithm. Non-cached tarballs were regenerated on demand, and all hell broke loose: https://github.blog/2023-02-21-update-on-the-future-stability-of-source-code-archives-and-hashes/ https://github.blog/2023-02-21-update-on-the-future-stabilit...
        Aissen 3y ago
        It was not the first instance of this happening; other times I'm not certain it was git's fault.
      - Lockal 3y ago
        Thank you for highlighting this. I've started a new discussion https://github.com/orgs/community/discussions/116557 https://github.com/orgs/community/discussions/116557 to provide strong guarantees for checksum stability for autogenerated tarballs attached to releases.
  - deleted 3y ago
    [deleted]
kijin 3y ago
What makes maintainers of major distros still rely on questionable tarballs to build packages? It's not as if these essential programs don't have a public, authoritative git repository. Is it because of inertia, because we've been using tarballs since before VCS was a thing? Is it to reduce the burden of package maintainership, by letting upstream do most of the transpiling and autotools plumbing work? Is it because some assets and test data are only included in the tarballs? Why are they not committed, either to the same repo or (if upstream wishes to keep the main repo small) some other repo? People have been calling for reproducible builds for years now, but reproducible builds don't mean anything if they are not built from authoritative sources with immutable commit IDs.
- Kluggy 3y ago
  It's absolutely because of inertia. Cutting a tarball is a very public indicator that the software is ready to be used. A git tag or branch doesn't have the clarity of "use this" as a release tarball does. That said, this is absolutely going to be changing now. We obviously can't keep relying on tarballs anymore. We'll find a new normal that will work for a very long time until some other critical issue arises and the cycle repeats.
  - kzrdude 3y ago
    We can use tarballs - they are useful as signed artifacts, but only if we verify that they are reproducible.
    - lupusreal 3y ago
      If you're going to download both the tarball and the got repo and verify the reproducibility of the tarball, then why bother with the tarball at all? You already have the got repo.
      - b112 3y ago
        A git repo can vanish overnight. Git is often used to snag source, but tarballs are still crafted after even then: https://news.ycombinator.com/item?id=39903813 https://news.ycombinator.com/item?id=39903813
andersa 3y ago
Why is this a thing? Can't packages use specific tags from the git repo? It seems so incredibly stupid to allow this, throwing out all of the "oh but it's open source you can review it" arguments in one go if the source displayed on GitHub is not what ends up used...
- atlasduo 3y ago
  Tags can be changed. To actually pin the source code revision, they should pin to a specific commit hash.
  - viraptor 3y ago
    Sometimes they get changed for a silly reason (someone just didn't think of the consequences). Sometimes they get changed because they have to - if I remember correctly it was Asterisk that needed to drop some copyrighted music samples and retagged old versions.
    - gunapologist99 3y ago
      That shouldn't matter, then; it should just result in a new release.
      - viraptor 3y ago
        You can't distribute the old versions with copyright violation. The problem doesn't go away just because you released a new version.
        nilamo 3y ago
        But then you have two different things (with and without copyrighted data) both with the same version. Wouldn't it be better to just remove the bad release, and publish a new one? They are, after all, not the same things, and shouldn't have the same release.
  - andersa 3y ago
sam_goody 3y ago
This has really been bugging me about npm. Anyone can publish a open source repo and add it to npmjs. Users going to the page on npmjs will see that the repo with the code is github.com/myrepo. But when I do `npm i myrepo` there is no guarantee that what is being pulled is in any way similar to what is in the linked repo. Creating a false feeling that the code could be reviewed. At the very least, Github should not allow this for code on their platform (ie, if npmjs has it listed as the repo of a project they should scan that the project actually builds to the content or notify npm who should have it flagged). Or npmjs should regularly scan the same - checking that the code you would get by compiling matches the code being offered Bear in mind that if a bad actor can get in at the level of the NPM user, even if the user is not running with elevated privileges (which it often is since you need to be a superuser to listen on port 80 or to read SSL certificates, and PM2 with handling for SSL is way above the ability of many devs, sigh), they can scan for vulnerabilities, and perhaps open themselves a very big hole.
- 3np 3y ago
  FYI: `npm publish` reports the current commit hash, if any, and this is publically viewable from the npm registry. This should help audit/verify if publishers have published what they say and if builds are reproducible. E.g. $ curl -sL https://registry.npmjs.com/colors | jq -r '.versions|map([.version, .gitHead]|@tsv)|.[]'
- PaoloBarbolini 3y ago
  Same for Rust. In the short term we're trying to solve it on the user's side with https://crates.io/crates/cargo-goggles https://crates.io/crates/cargo-goggles, but in the long term the registry should probably do it.
  - pabs3 3y ago
    Are crate maintainers going to accept having to do this? I've seen crates where the code in the crate was generated by a Python script not in the crate but only in git.
- bdd8f1df777b 3y ago
kzrdude 3y ago
I've enabled "trusted publishing" as it is called for python packages (publishing to cheeseshop/PyPI). However, what they call trusted publishing is just a configuration where PyPI tells github that publishing from a particular workflow name on a particular repository is ok, without further tokens. So PyPI "trusts" github actions and the maintainer is out of the loop. All good? Well, if you trust Github! It would be a lot better to me if both the maintainer and github were involved, something like - the maintainer signs off on some artifacts, and the github action verifies they are reproducible exactly by a workflow, and /then/ it's published.
- londons_explore 3y ago
  I do trust github with anything where any dishonesty on their part would be discoverable from the end result. Just a single piece of strong evidence that github the company had injected malware into something (rather than a github user) would end their business.
  - perihelions 3y ago
    I think we should have learned this lesson by now. The revelation that GitHub deliberately injects malware into things would be unlikely to "end their business": it would contentiously divide their userbase between purists who believe repo hosts should not be injecting malware into people's FOSS projects, and pragmatists who defend the trust-and-safety reasons for injecting malware into FOSS (say, to assist US intelligence preventing terror attacks). We've had this recurring discussion in many forms, and the typical result is that peoples' capacity for outrage wears out quickly. Who still remembers, for instance, that the GitHub entity sabotaged its users by giving the NSA backdoors to its private infrastructure [0]? How much business is Microsoft missing out on today, in 2024, because of that betrayal of trust? Essentially none, right? They're bigger than ever. GitHub sneaking NSA malware onto their platform—maybe selectively serving it to certain targets—would be perfectly in character with their past behavior. It would be smart business sense (they're a major DoD vendor). It'd be totally irrational to have a faith-based trust that they wouldn't do this. (There's a precedent: the FOSS host SourceForge destroyed itself by making the error of judgement of injecting code into people's repos [1]. But that was a decade ago: the Overton Window's shifted a lot. What SourceForge lost their business for doing in 2013—injecting ads—is something Microsoft comfortably gets away with today, on Windows desktops). [0] https://news.ycombinator.com/item?id=6027779 https://news.ycombinator.com/item?id=6027779 ("How Microsoft handed the NSA access to encrypted messages" (2013)) [1] https://en.wikipedia.org/wiki/SourceForge#Adware_controversy https://en.wikipedia.org/wiki/SourceForge#Adware_controversy
deleted 3y ago
[deleted]
shp0ngle 3y ago
Does Nix/Guix solve this? I have been skeptical of "rewrite everything into rust", but... maybe we should at least rewrite everything into Nix?
- 3836293648 3y ago
  No. It makes sure its inputs are unchanged, but you can still do anything impure within a buildscript called by nix. It would fix the tarball differing from upstream, but it would still allow the patch to be applied during build
- tomthehero 3y ago
  Please don't. Nix UX sucks big time. I don't know about Guix, but I'm pretty darn sure that anything else must be better than Nix cuz Nix is the rock bottom.
  - blueflow 3y ago
    Please elaborate. If i have more substantial facts about it then i might be more successful in convincing my CTO that its a bad idea.
  - forgotpwd16 3y ago
    Nix UX may suck (doesn't for me although can have some improvements) but Nix as concept/system is still good.
- blueflow 3y ago
  No. Nix pulls in tarballs/sources like any other package build system.
  - api 3y ago
    Computers have to run software that has to come from somewhere. I’ve been expecting a supply chain apocalypse for some time now given that the Internet has become a dark forest.
  - 1una 3y ago
jiggawatts 3y ago
I've noted similar vulnerabilities in every common package management system, including NuGet, Cargo, and NPM. There are giant gaps in security a truck could be driven through. Collectively, we must start taking the "bill of materials" that goes into our software much more seriously, including the chain of custody from source to binary. Start with linking each package directly to the Git commit hash it came from. Then a good start would be enforced reproducible builds. Don't trust the uploader to package their code. Run this step in the package management system from the provided source hash. Pure functional build systems would be a heck of a lot better, with no side effects of any kind permitted. No arbitrary scripts or arbitrary code during build. No file reads, network access, or API calls of any kind outside of the compiler toolkit itself. Then, I would like to see packages go through specially instrumented compilers that generate an "ingredient label" for each package. Transitive dependencies. Uses of unsafe constructs in otherwise safe languages. Lists of system calls used. Etc... You're about to say that clever hackers can work around that by using dynamic calling, or call-by-name, or some other clever trick. Sure, that's possible, but then the system can report that. If some random image codec or compression library uses any platform API at all, then it's instantly suspect. These things should be as pure as the driven snow: Bytes in, decoded data out.
- bandrami 3y ago
  > outside of the compiler toolkit itself Which toolchain includes, unfortunately, xz
  - pabs3 3y ago
    You can avoid that by bootstrapping the compiler and everything else from scratch. See bootstrappable.org and GNU Guix, which starts at a few hundred bytes of machine code and goes up to a full distro. https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/ https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-...
    - bandrami 3y ago
      IIRC that first GUIX build is at least theoretically the first Unix-like OS that was not rooted by Ken Thompson.
gunapologist99 3y ago
We could create and hash the tarball from git and compare it to the released tarball, even if it wasn't originally signed. We could even use Merkle trees (perhaps combining with find) to ensure that individual files were unchanged. At least then we could verify that the tarball was derived from that exact git commit.
zzzeek 3y ago
so glad pypi actively removed support for package signatures and even sends you an annoying email if your scripts dare to actually upload a signature anyway
userbinator 3y ago
What clickbait. I don't see anything actually "questionable" here, other than someone trying to ride on the xz hype train.
- Ajedi32 3y ago
  Not "questionable" in the sense that it might be malicious; the author acknowledges the changes are benign. But it's still questionable that the tarball made available to distros differs significantly from what's in the tagged release in git and nobody noticed that until now. As the xz incident has demonstrated, there are plenty of good security related reasons why that shouldn't be possible.
deleted 3y ago
[deleted]
cchance 3y ago
What a bullshit story

170 points by nateb2022 3y ago

dhx 3y ago

1.5.29-rc2 was tagged on 9 Nov 2023 [1] and, as an example, did not contain "N_("CJK Unified Ideographs Extension I")," in src/ibusunicodegen.h [2]. Commit 228f0a77b2047ade54e132bab69c0c03f0f41aae from 28 Feb 2023 introduced this change instead. It's the same person who tagged 1.5.29-rc2 and committed 228f0a77b2047ade54e132bab69c0c03f0f41aae which is typically an indication the maintainer tar'd their checked out git folder and accidentally included changes not get committed. The question raised is whether anyone is auditing these differences before the checksummed tarballs are added to package repositories. [1] https://github.com/ibus/ibus/releases/tag/1.5.29-rc2 https://github.com/ibus/ibus/releases/tag/1.5.29-rc2 [2] https://github.com/ibus/ibus/blob/0ad8e77bd36545974ad8acd0a5283cf72bc7c8ad/src/ibusunicodegen.h https://github.com/ibus/ibus/blob/0ad8e77bd36545974ad8acd0a5... [3] https://github.com/ibus/ibus/commit/228f0a77b2047ade54e132bab69c0c03f0f41aae https://github.com/ibus/ibus/commit/228f0a77b2047ade54e132ba...

dhx 3y ago
^ typo: 28 Feb 2023 is meant to be 28 Feb 2024, or almost 4 months later.
juitpykyk 3y ago
GitHub has the feature of downloading the tree as a zip, why is this not used?
- dhx 3y ago
  For many projects, the release tarballs only contain the files necessary to build the software, and not the following items that may be present in the same repository: - scripts used by project developers to import translations from another system and commit those translation changes to the repository - scripts to build release tarballs and sign them - continuous integration scripts - configuration and scripts used to setup a developer IDE environment
  - juitpykyk 3y ago
    What would be the problem of downloading a few MB more, aren't these source tarballs just used to build the distro binary and then they are deleted?
    - dhx 3y ago
      As this backdoor has shown, extra unnecessary files in the source files can make it easier to hide malicious code. If you take Gentoo as an example, when a software package is built, Gentoo creates a sandboxed environment first, disallowing the build process from impacting the rest of the operating system.[1] Removing superfluous files from the source tarballs minimises the ability for an attacker to get malicious code inside the sandboxed build environment. Sandboxes for building software are commonly used throughout Linux distributions, but I am unsure how strict those sandboxes are in general e.g. whether they use seccomp and really tighten what a build script can get up to. At least on Gentoo, there is a subset of packages (such as GNU coreutils) that are always just assumed to be needed to build software and they're always present in the sandbox. Build dependencies aren't as granular as "this build needs to use awk but not sed". [1] https://wiki.gentoo.org/wiki/Sandbox_(Portage) https://wiki.gentoo.org/wiki/Sandbox_(Portage)
  - roywashere 3y ago
    You can use .gitattributes export-ignore to influence what gets into the tarballs and what stays into the repository! It's super powerful but not often used
    - kzrdude 3y ago
      And export-subst to insert the current tag or git revision into the archive too. In fact export-subst is powerful enough that there is probably some way to create an exploit triggered by a particular payload inside a commit or tag message? :) Maybe not triggered, but it could be part of the chain.
    - bdd8f1df777b 3y ago
      I smell a new backdooring opportunity. Modifying .gitattributes to surreptitiously sneak some binary files into the GitHub release tarballs. Few poeple would take a look at .gitattributes.
- rwmj 3y ago
  There are two main reasons, one bad, one good(-ish): (1) Traditional autoconf assumed you only had a shell and cc and make, and so the ./configure script which is a huge ball of obscure shell commands is shipped. I think most distros will now delete and recreate these files, which probably should have happened a lot earlier. (Debian has been mostly doing this right for a long time already.) (2) Some programs generate a lot of code (eg. in libnbd we generate thousands of lines of boilerplate C from API descriptions*). To avoid people needing to install the specific tools that we use to generate that code, we distribute the generated files in the tarball, but it's not present in git. You can still build from git directly, and you can also verify the generated code exactly matches the tarball, but both cases mean extra build dependencies for end users and packagers. * Generating boilerplate code is a good thing in general as it reduces systematic errors, which are a vastly more common source of bugs compared to highly targeted supply chain attacks.
  - efrecon 3y ago
    Downloading the tarball/zip with just a shell and regular utils is possible. See https://github.com/efrecon/ungit https://github.com/efrecon/ungit
  - watt 3y ago
    I advocate for checking in the auto-generated code. You can see the differences between the tool runs, can see how changes in tooling affect the generated code, can see what might have caused a regression (hey it happens). Sometimes tooling can generate unstable files, I recall there was time when Eclipse was notorious there, for example when saving XML files they liked to reorder all the attributes. But these are bugs that need to be fixed. Tooling should generate perfectly reproducible files.
    - rwmj 3y ago
      We started off doing this, but you end up with enormous diffs which are themselves confusing. Example, only about 5% of this change is non-generated: https://github.com/libguestfs/libguestfs/commit/5186251f8f681f2ebb028423bb49a748861fd11e https://github.com/libguestfs/libguestfs/commit/5186251f8f68... Probably depends on the project as to whether this is feasible, but for us we intentionally want to generate everything we can in order to reduce systematic errors.
- Lockal 3y ago
  There is a minor note that technically there is a weak guarantee that checksums won't break after server update and recompression with different version / alternative implementation of gzip. https://github.com/orgs/community/discussions/45830 https://github.com/orgs/community/discussions/45830
  - Aissen 3y ago
    It's not a minor note, it's a major reason that the github auto-generated tarballs are useless as-is, since they are not stable.
    - rwmj 3y ago
      github have (for the moment) backed down and currently the auto-generated tarballs are stable, but they have in the past and may in the future change this.
    - colejohnson66 3y ago
      This was not GitHub’s fault, but Git itself combined with cache pruning. Specifically, GitHub updating to Git 2.38 which changed the algorithm. Non-cached tarballs were regenerated on demand, and all hell broke loose: https://github.blog/2023-02-21-update-on-the-future-stability-of-source-code-archives-and-hashes/ https://github.blog/2023-02-21-update-on-the-future-stabilit...
      - Aissen 3y ago
        It was not the first instance of this happening; other times I'm not certain it was git's fault.
    - Lockal 3y ago
      Thank you for highlighting this. I've started a new discussion https://github.com/orgs/community/discussions/116557 https://github.com/orgs/community/discussions/116557 to provide strong guarantees for checksum stability for autogenerated tarballs attached to releases.
- deleted 3y ago
  [deleted]

kijin 3y ago

What makes maintainers of major distros still rely on questionable tarballs to build packages? It's not as if these essential programs don't have a public, authoritative git repository. Is it because of inertia, because we've been using tarballs since before VCS was a thing? Is it to reduce the burden of package maintainership, by letting upstream do most of the transpiling and autotools plumbing work? Is it because some assets and test data are only included in the tarballs? Why are they not committed, either to the same repo or (if upstream wishes to keep the main repo small) some other repo? People have been calling for reproducible builds for years now, but reproducible builds don't mean anything if they are not built from authoritative sources with immutable commit IDs.

Kluggy 3y ago
It's absolutely because of inertia. Cutting a tarball is a very public indicator that the software is ready to be used. A git tag or branch doesn't have the clarity of "use this" as a release tarball does. That said, this is absolutely going to be changing now. We obviously can't keep relying on tarballs anymore. We'll find a new normal that will work for a very long time until some other critical issue arises and the cycle repeats.
- kzrdude 3y ago
  We can use tarballs - they are useful as signed artifacts, but only if we verify that they are reproducible.
  - lupusreal 3y ago
    If you're going to download both the tarball and the got repo and verify the reproducibility of the tarball, then why bother with the tarball at all? You already have the got repo.
    - b112 3y ago
      A git repo can vanish overnight. Git is often used to snag source, but tarballs are still crafted after even then: https://news.ycombinator.com/item?id=39903813 https://news.ycombinator.com/item?id=39903813

andersa 3y ago

Why is this a thing? Can't packages use specific tags from the git repo? It seems so incredibly stupid to allow this, throwing out all of the "oh but it's open source you can review it" arguments in one go if the source displayed on GitHub is not what ends up used...

atlasduo 3y ago
Tags can be changed. To actually pin the source code revision, they should pin to a specific commit hash.
- viraptor 3y ago
  Sometimes they get changed for a silly reason (someone just didn't think of the consequences). Sometimes they get changed because they have to - if I remember correctly it was Asterisk that needed to drop some copyrighted music samples and retagged old versions.
  - gunapologist99 3y ago
    That shouldn't matter, then; it should just result in a new release.
    - viraptor 3y ago
      You can't distribute the old versions with copyright violation. The problem doesn't go away just because you released a new version.
      - nilamo 3y ago
        But then you have two different things (with and without copyrighted data) both with the same version. Wouldn't it be better to just remove the bad release, and publish a new one? They are, after all, not the same things, and shouldn't have the same release.
- andersa 3y ago

sam_goody 3y ago

This has really been bugging me about npm. Anyone can publish a open source repo and add it to npmjs. Users going to the page on npmjs will see that the repo with the code is github.com/myrepo. But when I do `npm i myrepo` there is no guarantee that what is being pulled is in any way similar to what is in the linked repo. Creating a false feeling that the code could be reviewed. At the very least, Github should not allow this for code on their platform (ie, if npmjs has it listed as the repo of a project they should scan that the project actually builds to the content or notify npm who should have it flagged). Or npmjs should regularly scan the same - checking that the code you would get by compiling matches the code being offered Bear in mind that if a bad actor can get in at the level of the NPM user, even if the user is not running with elevated privileges (which it often is since you need to be a superuser to listen on port 80 or to read SSL certificates, and PM2 with handling for SSL is way above the ability of many devs, sigh), they can scan for vulnerabilities, and perhaps open themselves a very big hole.

3np 3y ago
FYI: `npm publish` reports the current commit hash, if any, and this is publically viewable from the npm registry. This should help audit/verify if publishers have published what they say and if builds are reproducible. E.g. $ curl -sL https://registry.npmjs.com/colors | jq -r '.versions|map([.version, .gitHead]|@tsv)|.[]'
PaoloBarbolini 3y ago
Same for Rust. In the short term we're trying to solve it on the user's side with https://crates.io/crates/cargo-goggles https://crates.io/crates/cargo-goggles, but in the long term the registry should probably do it.
- pabs3 3y ago
  Are crate maintainers going to accept having to do this? I've seen crates where the code in the crate was generated by a Python script not in the crate but only in git.
bdd8f1df777b 3y ago

kzrdude 3y ago

I've enabled "trusted publishing" as it is called for python packages (publishing to cheeseshop/PyPI). However, what they call trusted publishing is just a configuration where PyPI tells github that publishing from a particular workflow name on a particular repository is ok, without further tokens. So PyPI "trusts" github actions and the maintainer is out of the loop. All good? Well, if you trust Github! It would be a lot better to me if both the maintainer and github were involved, something like - the maintainer signs off on some artifacts, and the github action verifies they are reproducible exactly by a workflow, and /then/ it's published.

londons_explore 3y ago
I do trust github with anything where any dishonesty on their part would be discoverable from the end result. Just a single piece of strong evidence that github the company had injected malware into something (rather than a github user) would end their business.
- perihelions 3y ago
  I think we should have learned this lesson by now. The revelation that GitHub deliberately injects malware into things would be unlikely to "end their business": it would contentiously divide their userbase between purists who believe repo hosts should not be injecting malware into people's FOSS projects, and pragmatists who defend the trust-and-safety reasons for injecting malware into FOSS (say, to assist US intelligence preventing terror attacks). We've had this recurring discussion in many forms, and the typical result is that peoples' capacity for outrage wears out quickly. Who still remembers, for instance, that the GitHub entity sabotaged its users by giving the NSA backdoors to its private infrastructure [0]? How much business is Microsoft missing out on today, in 2024, because of that betrayal of trust? Essentially none, right? They're bigger than ever. GitHub sneaking NSA malware onto their platform—maybe selectively serving it to certain targets—would be perfectly in character with their past behavior. It would be smart business sense (they're a major DoD vendor). It'd be totally irrational to have a faith-based trust that they wouldn't do this. (There's a precedent: the FOSS host SourceForge destroyed itself by making the error of judgement of injecting code into people's repos [1]. But that was a decade ago: the Overton Window's shifted a lot. What SourceForge lost their business for doing in 2013—injecting ads—is something Microsoft comfortably gets away with today, on Windows desktops). [0] https://news.ycombinator.com/item?id=6027779 https://news.ycombinator.com/item?id=6027779 ("How Microsoft handed the NSA access to encrypted messages" (2013)) [1] https://en.wikipedia.org/wiki/SourceForge#Adware_controversy https://en.wikipedia.org/wiki/SourceForge#Adware_controversy

deleted 3y ago

[deleted]

shp0ngle 3y ago

Does Nix/Guix solve this? I have been skeptical of "rewrite everything into rust", but... maybe we should at least rewrite everything into Nix?

3836293648 3y ago
No. It makes sure its inputs are unchanged, but you can still do anything impure within a buildscript called by nix. It would fix the tarball differing from upstream, but it would still allow the patch to be applied during build
tomthehero 3y ago
Please don't. Nix UX sucks big time. I don't know about Guix, but I'm pretty darn sure that anything else must be better than Nix cuz Nix is the rock bottom.
- blueflow 3y ago
  Please elaborate. If i have more substantial facts about it then i might be more successful in convincing my CTO that its a bad idea.
- forgotpwd16 3y ago
  Nix UX may suck (doesn't for me although can have some improvements) but Nix as concept/system is still good.
blueflow 3y ago
No. Nix pulls in tarballs/sources like any other package build system.
- api 3y ago
  Computers have to run software that has to come from somewhere. I’ve been expecting a supply chain apocalypse for some time now given that the Internet has become a dark forest.
- 1una 3y ago

jiggawatts 3y ago

I've noted similar vulnerabilities in every common package management system, including NuGet, Cargo, and NPM. There are giant gaps in security a truck could be driven through. Collectively, we must start taking the "bill of materials" that goes into our software much more seriously, including the chain of custody from source to binary. Start with linking each package directly to the Git commit hash it came from. Then a good start would be enforced reproducible builds. Don't trust the uploader to package their code. Run this step in the package management system from the provided source hash. Pure functional build systems would be a heck of a lot better, with no side effects of any kind permitted. No arbitrary scripts or arbitrary code during build. No file reads, network access, or API calls of any kind outside of the compiler toolkit itself. Then, I would like to see packages go through specially instrumented compilers that generate an "ingredient label" for each package. Transitive dependencies. Uses of unsafe constructs in otherwise safe languages. Lists of system calls used. Etc... You're about to say that clever hackers can work around that by using dynamic calling, or call-by-name, or some other clever trick. Sure, that's possible, but then the system can report that. If some random image codec or compression library uses any platform API at all, then it's instantly suspect. These things should be as pure as the driven snow: Bytes in, decoded data out.

bandrami 3y ago
> outside of the compiler toolkit itself Which toolchain includes, unfortunately, xz
- pabs3 3y ago
  You can avoid that by bootstrapping the compiler and everything else from scratch. See bootstrappable.org and GNU Guix, which starts at a few hundred bytes of machine code and goes up to a full distro. https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/ https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-...
  - bandrami 3y ago
    IIRC that first GUIX build is at least theoretically the first Unix-like OS that was not rooted by Ken Thompson.

gunapologist99 3y ago

We could create and hash the tarball from git and compare it to the released tarball, even if it wasn't originally signed. We could even use Merkle trees (perhaps combining with find) to ensure that individual files were unchanged. At least then we could verify that the tarball was derived from that exact git commit.

zzzeek 3y ago

so glad pypi actively removed support for package signatures and even sends you an annoying email if your scripts dare to actually upload a signature anyway

userbinator 3y ago

What clickbait. I don't see anything actually "questionable" here, other than someone trying to ride on the xz hype train.

Ajedi32 3y ago
Not "questionable" in the sense that it might be malicious; the author acknowledges the changes are benign. But it's still questionable that the tarball made available to distros differs significantly from what's in the tagged release in git and nobody noticed that until now. As the xz incident has demonstrated, there are plenty of good security related reasons why that shouldn't be possible.

deleted 3y ago

[deleted]

cchance 3y ago

What a bullshit story