5 ms·

Maybe it's time to move away from xz which has been criticized in the past as a not great format, has had maintainer issues for years, and now turns out to be i

by jgb1984 3y ago

Maybe it's time to move away from xz which has been criticized in the past as a not great format, has had maintainer issues for years, and now turns out to be infiltrated with security problems. Moving to zstd seems worthwhile.

angiosperm 3y ago
Best would be to disable anything in sshd that can perform compression or decompression. There is no value in that capability, and way too much risk.
- lifthrasiir 3y ago
  Agreed on the SSH compression, but it would be not relevant to this particular vulnerability because liblzma was never used in the SSH compression anyway.
  - deleted 3y ago
    [deleted]
  - cpach 3y ago
    I don’t really understand the connection between liblzma and sshd, it had something do with systemd, or…?
    - lifthrasiir 3y ago
      Apparently liblzma in the compromised tarball would search for sshd functions and replace them with their own versions. This normally doesn't happen for the reason I've described, but many distros tend to compile sshd with a plugin that makes use of libsystemd, which depends on liblzma and therefore sshd will be linked with liblzma in that case. Liblzma could have done anything else it wanted, so the limited reach of this vulnerability is actually a better scenario...
    - dboreham 3y ago
      Yes.
    - cesarb 3y ago
      The connection is that sshd calls code in libsystemd when running as a systemd service, to notify systemd that it finished starting. An unrelated functionality in libsystemd (my guess is that it's related to parsing journald logs, since they can use LZMA compression) uses the lzma library, so libsystemd is linked against it. That is, the chain is: sshd is linked against libsystemd, which is linked against liblzma. When loading the sshd executable, the dynamic linker will load all these libraries. The backdoor was added to initialization code in liblzma (which runs whenever the dynamic linker loads that library). So that malicious code runs before sshd starts executing, and manipulates the dynamic linker so that it will do something (which is still being analyzed AFAIK) when some specific functions are called by the sshd executable.
      - angiosperm 3y ago
        Wow, thank you! So it is wrong for such a fundamental service as systemd to depend on such complicated processes ... just as the anti-systemd people said. Except of course shellscripts and /bin programs are just as complicated, when you check.
begueradj 3y ago
We could say something similar about other tools. Even something like OpenSSL had a severe security issue (Heartbleed) discovered on 2014 and which was introduced 2 years earlier by the maintainer himself through a Git commit which occurred, if I remember, on the middle of the night of 31.12.2011 or 01.01.2012 (which lead some people to speculate whether that was intentional since "everyone" is usually celebrating with his/her family at that time of the year)
- Maxious 3y ago
  People did indeed move to LibreSSL and s2n-tls after that one incident
  - cpach 3y ago
    And later on crypto/tls, etc.
- liquidpele 3y ago
  The maintainer merged the patch, but the code change came from someone else IIRC.
pmontra 3y ago
It's more or less what the current last comment on the Debian thread is wondering about "Is xz-utils going to be maintained? Will we want to keep in the archive an unmaintained low-level library - low-level as in, susceptible of getting pulled as a dependency in lots of places - and rely on it for components such as dpkg?"
jl6 3y ago
The format is fine, and zstd would be a downgrade in terms of compression ratio (which is the main reason xz is chosen for its current use cases). The lesson to learn here is not about making technical improvements, it’s to do something about improving the way we manage and maintain core open source infrastructure. Ref XKCD 2347.
- lifthrasiir 3y ago
  No, xz was chosen mainly because zstandard didn't exist at that time [1]. Zstandard does almost completely replace those "expensive" compression formats by providing a comparable compression ratio but being much faster. [1] https://lists.debian.org/debian-devel-announce/2011/08/msg00001.html https://lists.debian.org/debian-devel-announce/2011/08/msg00...
  - jl6 3y ago
    "almost" and "comparable", but xz (lzma) still compresses at a higher ratio, which is still a highly desirable feature. If the decision was being made today, a good faith debate could be had over whether the distro packaging use case benefits more from faster decompression or smaller file size, but I suspect that, globally, the average PC can decompress xz faster than the average broadband connection can download it.
    - lifthrasiir 3y ago
      I will assume that the average effective broadband connection is about 20 MB/s, and both xz and zstd decompressions are faster than the download under this assumption. But there are lesser known compression algorithms that are more efficient in terms of compression ratio for this target decompression speed, like zpaq [1], which are rarely considered as alternatives. So multiple factors exist here and I believe zstd excels at pretty much every other factor compared to xz, making it a very likely choice. [1] https://mattmahoney.net/dc/zpaq.html https://mattmahoney.net/dc/zpaq.html
- pgeorgi 3y ago
  The format is fine, except for https://www.nongnu.org/lzip/xz_inadequate.html https://www.nongnu.org/lzip/xz_inadequate.html

angiosperm 3y ago

Best would be to disable anything in sshd that can perform compression or decompression. There is no value in that capability, and way too much risk.

lifthrasiir 3y ago
Agreed on the SSH compression, but it would be not relevant to this particular vulnerability because liblzma was never used in the SSH compression anyway.
- deleted 3y ago
  [deleted]
- cpach 3y ago
  I don’t really understand the connection between liblzma and sshd, it had something do with systemd, or…?
  - lifthrasiir 3y ago
    Apparently liblzma in the compromised tarball would search for sshd functions and replace them with their own versions. This normally doesn't happen for the reason I've described, but many distros tend to compile sshd with a plugin that makes use of libsystemd, which depends on liblzma and therefore sshd will be linked with liblzma in that case. Liblzma could have done anything else it wanted, so the limited reach of this vulnerability is actually a better scenario...
  - dboreham 3y ago
    Yes.
  - cesarb 3y ago
    The connection is that sshd calls code in libsystemd when running as a systemd service, to notify systemd that it finished starting. An unrelated functionality in libsystemd (my guess is that it's related to parsing journald logs, since they can use LZMA compression) uses the lzma library, so libsystemd is linked against it. That is, the chain is: sshd is linked against libsystemd, which is linked against liblzma. When loading the sshd executable, the dynamic linker will load all these libraries. The backdoor was added to initialization code in liblzma (which runs whenever the dynamic linker loads that library). So that malicious code runs before sshd starts executing, and manipulates the dynamic linker so that it will do something (which is still being analyzed AFAIK) when some specific functions are called by the sshd executable.
    - angiosperm 3y ago
      Wow, thank you! So it is wrong for such a fundamental service as systemd to depend on such complicated processes ... just as the anti-systemd people said. Except of course shellscripts and /bin programs are just as complicated, when you check.

begueradj 3y ago

We could say something similar about other tools. Even something like OpenSSL had a severe security issue (Heartbleed) discovered on 2014 and which was introduced 2 years earlier by the maintainer himself through a Git commit which occurred, if I remember, on the middle of the night of 31.12.2011 or 01.01.2012 (which lead some people to speculate whether that was intentional since "everyone" is usually celebrating with his/her family at that time of the year)

Maxious 3y ago
People did indeed move to LibreSSL and s2n-tls after that one incident
- cpach 3y ago
  And later on crypto/tls, etc.
liquidpele 3y ago
The maintainer merged the patch, but the code change came from someone else IIRC.

pmontra 3y ago

It's more or less what the current last comment on the Debian thread is wondering about "Is xz-utils going to be maintained? Will we want to keep in the archive an unmaintained low-level library - low-level as in, susceptible of getting pulled as a dependency in lots of places - and rely on it for components such as dpkg?"

jl6 3y ago

The format is fine, and zstd would be a downgrade in terms of compression ratio (which is the main reason xz is chosen for its current use cases). The lesson to learn here is not about making technical improvements, it’s to do something about improving the way we manage and maintain core open source infrastructure. Ref XKCD 2347.

lifthrasiir 3y ago
No, xz was chosen mainly because zstandard didn't exist at that time [1]. Zstandard does almost completely replace those "expensive" compression formats by providing a comparable compression ratio but being much faster. [1] https://lists.debian.org/debian-devel-announce/2011/08/msg00001.html https://lists.debian.org/debian-devel-announce/2011/08/msg00...
- jl6 3y ago
  "almost" and "comparable", but xz (lzma) still compresses at a higher ratio, which is still a highly desirable feature. If the decision was being made today, a good faith debate could be had over whether the distro packaging use case benefits more from faster decompression or smaller file size, but I suspect that, globally, the average PC can decompress xz faster than the average broadband connection can download it.
  - lifthrasiir 3y ago
    I will assume that the average effective broadband connection is about 20 MB/s, and both xz and zstd decompressions are faster than the download under this assumption. But there are lesser known compression algorithms that are more efficient in terms of compression ratio for this target decompression speed, like zpaq [1], which are rarely considered as alternatives. So multiple factors exist here and I believe zstd excels at pretty much every other factor compared to xz, making it a very likely choice. [1] https://mattmahoney.net/dc/zpaq.html https://mattmahoney.net/dc/zpaq.html
pgeorgi 3y ago
The format is fine, except for https://www.nongnu.org/lzip/xz_inadequate.html https://www.nongnu.org/lzip/xz_inadequate.html