4 ms·

How are the binary files passed to those stage-0 commands?

viraptor 3y ago
They already exist in the source. They're split in the compression test files themselves. (Unless you meant some other binaries?)
- jijijijij 3y ago
  Yeah, but how exactly are they passed to those commands? I don't see/understand that part. I don't see the "take this file here" part.
  - chaosite 3y ago
    That's for 2 reasons: 1. It might not be there in the place where you're looking. It exists in the m4 in the release tarballs, not in the git repo. 2. It's highly obfuscated.
    - jijijijij 3y ago
      No, as far as I understand the binary files must be pointed at here: '$gl_am_configmake' ... But I don't see how. This: 'gl_am_configmake=`grep -aErls "#{4}[[:alnum:]]{5}#{4}$" $srcdir/`' seem to match the '####Hello####', but, as far as I can see, that's supposed to be the already converted script?! I presumed the binary files not to contain human readable strings, maybe that's the whole confusion.
      - credulousperson 3y ago
        Opening bad-3-corrupt_lzma2.xz in an editor reveals it indeed has the string ####Hello####. I don't know enough about lzma compression streams to explain how this appears in the "compressed" version of the payload, but it does.
        loeg 3y ago
        I think part of it being a bad/corrupt test case means it doesn't have to be valid xz encoding. But I don't know if that even matters.
        jijijijij 3y ago
        Thanks. The riddle has been solved :) Do you have a (safe web view) version of those files? I would like to see what they look like to a casual observer. Judging by the 'tr' assembly command I would expect the bad-3-corrupt_ligma2.xz to be somewhat recognizable as script.
        deanishe 3y ago
        > I don't know enough about lzma compression streams to explain how this appears in the "compressed" version of the payload, but it does. From what I've read, the payload isn't stored in the archive, but rather the test file itself is a sandwich of xz data and payload: There are 1024 bytes of xz archive, N bytes of payload, another 1024 of xz, etc.
    - fomine3 3y ago
      m4 is somewhat obfuscated by default, that's a part of the problem IMO
      - jijijijij 3y ago
        Looks pretty much like bash to me. Which means... yeah.
deng 3y ago
The macro defined in build_to_host.m4 is probably called on the tests subdirectory, so it gets these files as a parameter. EDIT: It is called here and will do the extraction of the backdoor when run in the 'tests' subdirectory: https://salsa.debian.org/debian/xz-utils/-/blob/debian/unstable/m4/gettext.m4?ref_type=heads#L378 https://salsa.debian.org/debian/xz-utils/-/blob/debian/unsta... EDIT2: So it will get the directory as a parameter, the actual file is encoded indirectly here: https://salsa.debian.org/debian/xz-utils/-/blob/debian/unstable/m4/build-to-host.m4?ref_type=heads#L86 https://salsa.debian.org/debian/xz-utils/-/blob/debian/unsta... This grep will only match bad-3-corrupt_lzma2.xz
- jijijijij 3y ago
  Thanks. Yeah, below I learned the '####Hello####' string to be present in the "bad test file" (I haven't seen it myself). I was just not expecting a "binary" file to be basically a text file and thought the `grep` was matching post extraction somehow. That's the root of my confusion. I do understand now where the file gets localized. IIRC only the "binary" files where added secretly, right? But the build script was there for people to inspect? If so, I have to say, it's not that obfuscated, to someone who actually knew .m4, I guess. At least the grep line should have raised the question of why. I think, part of the problem is normalization of arcane, cryptic scripts in the first place, where people sign off on things they don't fully understand in the moment, since - c'mon - "knowledge" of these old gibberish scripting languages only lives transiently between Google searches and your working memory. Without looking it up, can you tell me what this does in bash: `echo $(. /tmp/file)` ? I think, I've seen at least one "xz backdoor detection script" by "someone trusted" in one of the xz threads here, which was at least as cryptic as the .m4 script, containing several `eval`s. I mean, you could probably throw your head onto the keyboard and there is a good chance it's valid bash or regex, or at least common bash can be indiscernible from random gibberish until you manually disassemble it, feeling smuck and dopaminergic. The condensed arcane wizardry around Linux (bash, autotools, CMake, ...) and C (macros, single letter variable culture, ...) is really fun in a way, but it's such a huge vulnerability in itself, before we even talk memory safety.
  - deng 3y ago

They already exist in the source. They're split in the compression test files themselves. (Unless you meant some other binaries?)

jijijijij 3y ago
Yeah, but how exactly are they passed to those commands? I don't see/understand that part. I don't see the "take this file here" part.
- chaosite 3y ago
  That's for 2 reasons: 1. It might not be there in the place where you're looking. It exists in the m4 in the release tarballs, not in the git repo. 2. It's highly obfuscated.
  - jijijijij 3y ago
    No, as far as I understand the binary files must be pointed at here: '$gl_am_configmake' ... But I don't see how. This: 'gl_am_configmake=`grep -aErls "#{4}[[:alnum:]]{5}#{4}$" $srcdir/`' seem to match the '####Hello####', but, as far as I can see, that's supposed to be the already converted script?! I presumed the binary files not to contain human readable strings, maybe that's the whole confusion.
    - credulousperson 3y ago
      Opening bad-3-corrupt_lzma2.xz in an editor reveals it indeed has the string ####Hello####. I don't know enough about lzma compression streams to explain how this appears in the "compressed" version of the payload, but it does.
      - loeg 3y ago
        I think part of it being a bad/corrupt test case means it doesn't have to be valid xz encoding. But I don't know if that even matters.
      - jijijijij 3y ago
        Thanks. The riddle has been solved :) Do you have a (safe web view) version of those files? I would like to see what they look like to a casual observer. Judging by the 'tr' assembly command I would expect the bad-3-corrupt_ligma2.xz to be somewhat recognizable as script.
      - deanishe 3y ago
        > I don't know enough about lzma compression streams to explain how this appears in the "compressed" version of the payload, but it does. From what I've read, the payload isn't stored in the archive, but rather the test file itself is a sandwich of xz data and payload: There are 1024 bytes of xz archive, N bytes of payload, another 1024 of xz, etc.
  - fomine3 3y ago
    m4 is somewhat obfuscated by default, that's a part of the problem IMO
    - jijijijij 3y ago
      Looks pretty much like bash to me. Which means... yeah.

deng 3y ago

The macro defined in build_to_host.m4 is probably called on the tests subdirectory, so it gets these files as a parameter. EDIT: It is called here and will do the extraction of the backdoor when run in the 'tests' subdirectory: https://salsa.debian.org/debian/xz-utils/-/blob/debian/unstable/m4/gettext.m4?ref_type=heads#L378 https://salsa.debian.org/debian/xz-utils/-/blob/debian/unsta... EDIT2: So it will get the directory as a parameter, the actual file is encoded indirectly here: https://salsa.debian.org/debian/xz-utils/-/blob/debian/unstable/m4/build-to-host.m4?ref_type=heads#L86 https://salsa.debian.org/debian/xz-utils/-/blob/debian/unsta... This grep will only match bad-3-corrupt_lzma2.xz

jijijijij 3y ago
Thanks. Yeah, below I learned the '####Hello####' string to be present in the "bad test file" (I haven't seen it myself). I was just not expecting a "binary" file to be basically a text file and thought the `grep` was matching post extraction somehow. That's the root of my confusion. I do understand now where the file gets localized. IIRC only the "binary" files where added secretly, right? But the build script was there for people to inspect? If so, I have to say, it's not that obfuscated, to someone who actually knew .m4, I guess. At least the grep line should have raised the question of why. I think, part of the problem is normalization of arcane, cryptic scripts in the first place, where people sign off on things they don't fully understand in the moment, since - c'mon - "knowledge" of these old gibberish scripting languages only lives transiently between Google searches and your working memory. Without looking it up, can you tell me what this does in bash: `echo $(. /tmp/file)` ? I think, I've seen at least one "xz backdoor detection script" by "someone trusted" in one of the xz threads here, which was at least as cryptic as the .m4 script, containing several `eval`s. I mean, you could probably throw your head onto the keyboard and there is a good chance it's valid bash or regex, or at least common bash can be indiscernible from random gibberish until you manually disassemble it, feeling smuck and dopaminergic. The condensed arcane wizardry around Linux (bash, autotools, CMake, ...) and C (macros, single letter variable culture, ...) is really fun in a way, but it's such a huge vulnerability in itself, before we even talk memory safety.
- deng 3y ago