7 ms·

Except you can't because hundreds of thousands of project have "log4j" in their dependency list not "wyldfire-fixed-log4j". If log4j broke and there was no main

by zaarn 5y ago

Except you can't because hundreds of thousands of project have "log4j" in their dependency list not "wyldfire-fixed-log4j". If log4j broke and there was no maintainer, a fork would not fix things.

blibble 5y ago
the documentation also needs to include the 5 line dependency exclusion that you place in your pom to exclude the bad version
- zaarn 5y ago
  And if you do that everytime an unmaintained project gets a 10 severity CVE, we'll have that at 200 lines in no time. Proabably petition whatever generates your Pom.xml to auto-include those lines.
mantaraygun 5y ago
> hundreds of thousands of project have "log4j" in their dependency list That's on them. They can migrate to wyldfire-fixed-log4j if they want, or to whatever other fork they want, or they can keep using the old busted shit. It's up to them, there's no problem here.
IiydAbITMvJkqKf 5y ago
Sure you can fix it, change the "log4j" in your dependency list to "wyldfire-fixed-log4j". You can't do that with closed source (either because you care about legalities, or because you have no way of obtaining a compilable and readable source code).
- zaarn 5y ago
  Until all hundreds of thousands of projects have changed that, it'll take time. That doesn't mention the fact you now also have to fork unmaintained projects relying on log4j to use the new version. If you're developing a major java projects, that's potentially thousands of dependencies you have to patch. I'm not suggesting that closed source is the solution, I'm saying the current system is not sustainable and neither is just forking projects. Because which fork do you follow, and how do you make sure all your dependencies follow the fork you want and not an unmaintained fork?
  - the_duke 5y ago
    Some package managers (eg yarn , cargo) have a solution for that and allow you to specify a local resolution override that will apply to all dependencies.
    - zaarn 5y ago
      But again, that means EVERY single product, open source or not, will have to patch their deployment to use the fork instead of simply using the updated version of upstream. Permanently no less if it's unmaintained. Meaning every single Java product would have to include that "if you want to use log4j, you have to include this patch line to use a fork, because of unmaintained dependencies". That just ossifies log4j and fixes nothing. Patching dependencies in Cargo is not meant to permanently replace code, it's to be used while you're waiting for an upstream merge.
      - IiydAbITMvJkqKf 5y ago
        You can fix it for your own applications and libraries. The tooling exists. You can't force other people to use your fork.
        zaarn 5y ago
        Correct, all those other people will have to manually find some fork or make their own, repeating the same work thousands of times and ossifying that some part of your java build system now permanently includes a section to patch log4j dependencies.
        alexvoda 5y ago
        You can not force your dependencies to do what you want. You can only patch the problem for yourself and maintain that patch in the build system, or you can fork each dependency in the chain, maintain those and then your build system is cleaner. There is nothing wrong with these options.
- kube-system 5y ago
  The problem is, my dependencies also have dependencies. If the problematic dependency is any more than one degree of separation removed, you're back to square one.
  - debug-desperado 5y ago
    A nightmare in Node world perhaps (our latest simple web server has over 2,000 dependencies). In the Java world it’s much simpler with a few excludes in your POM. Especially for libraries that keep stable interfaces, such as a logging library.
    - zaarn 5y ago
      How many excludes is Java going to ship is forking the problem of unmaintained libraries away becomes too much? When it reaches 5kb? 10kb? 100kb of excludes?
CuriousCosmic 5y ago
Strictly speaking in the Java space that's a perfectly reasonable assumption. It's a bit more work but you can just substitute the jar/modules for log4j following the build with the patched versions. At the end of the day it is just a bunch of class files in jars/zips which makes patching a lot easier.
- zaarn 5y ago
  Hot patching a jar is not a scalable solution in the long term, especially if you're considering that in a larger project there may be multiple unmaintained projects that now need permanent hotpatching.
  - CuriousCosmic 5y ago
    At least with gradle you can easily add some logic to the buildscript to substitute all instances of a specific dependency for a java library/application with a fork. It's only a handful of lines of code and it is technically a bit hacky but it's really no worse than any of the other jank you are forced to do when building any moderate to complex gradle based java project.
    - zaarn 5y ago
      And when another library is unmaintained and breaks, you add those few lines again. And again and again and again, until your gradle file is 90% hotpatching your dependencies.
      - CuriousCosmic 5y ago
        Yep but that's just the breaks. Especially with ecosystems that have tall dependency trees. Luckily those few lines can mostly be shared and you end up with a handful of lines and a map from old dep to new dep which makes it not too painful. This whole issue is why I don't like these ecosystems but at the very least in the Java space dealing with this issue isn't too painful. I can't speak for the JS space but this is also a pretty trivial fix in the Rust space and it isn't too painful with C++ or C projects. Nix (mostly used for Haskell but it's a pretty solid universal package manager) has some pretty good tooling for modifying upstream dependencies. Spack (package manager specialised for HPC and embedded projects) has really good support for this as well. My point being that people have run into these issues in the past a lot in certain spaces and as a result those spaces have come up with solutions for this problem a long time ago. It just so happens that for the most part the average dev doesn't see a lot of that because these issues get dealt with upstream or just aren't common in most industries. The embedded industry is particularly used to this considering how much of the kit from manufacturers is just outright wrong or broken 3 levels deep. The only reason I'm as well versed with this as I am is because I've been in the embedded space patching over broken HALs and I've bled my blood dealing with legacy enterprise applications that were initially written in Java 1.1 and haphazardly dragged along over the decades. TLDR: This is an ugly fix but ultimately it's the best ugly fix to an ugly problem and every software ecosystem eventually gets to deal with it once they get old and crusty enough.
blindmute 5y ago
Code doesn't just "break". It still works the same. If a vulnerability is discovered you either update the package or switch to a fork. Switching is not really that much extra work compared to updating. This issue has really nothing to do with open source except that you have the extra option of forking.
- zaarn 5y ago
  I argue that if your code gets assigned a 10/10 Severity CVE, then it broke, even if nothing changed. It just broke a while ago. Switching one library is not much work. But if it becomes the standard approach then the first time you fire up your java project you either spent 30 minutes to put in all the exclusions to provide your favorite forks of dead projects or your pom.xml will already be 5KB large on generation just for those exclusions and patches.
quickthrower2 5y ago
Thats the problem. If you use say React then you are using it as is without any warranty implied. React is the equivalent of an aircraft in complexity when looking at its dependencies and then the browser stack itself on top! React gives you a free copy of an aircraft and will probably maintain it for years. But maybe they rely on an altimeter developed by a poor person who decides they dont have time to work on it because they need to help their mum. Well thats everyone elses problem now. but the ease of download and use of React and other code libraries creates a sense of entitlement. Yes if its broke and you need it fixed you have to pay someone now! Its still libre its just not gratis anymore
dopkew 5y ago
Off-topic but very important: The widespread misuse of the word 'dependency' is very very icky, linguistically speaking. This word originally meant: 'in a state of being dependent'. Usage: "Ronny hated his dependency on the kindness of strangers.". But it's being used to refer to the 'dependent', as well as the 'dependee', which is very poor form: https://english.stackexchange.com/questions/25575/what-is-the-correct-word-for-dependee https://english.stackexchange.com/questions/25575/what-is-th... https://wikidiff.com/dependee/dependent https://wikidiff.com/dependee/dependent
- zaarn 5y ago
  I don't think that is any issue whatsoever, the software engineering world has largely agreed on the usage of the word dependency, doesn't really matter what linguists think. Language is consensus, not thesis papers.
- pritambaral 5y ago
  The original meaning still works. In the software world, where abstractions abound, each dependee of yours can be said to add one 'state of being dependent' to your set of requirements. That's what it meant originally in this young field; when an author chooses to use a library instead of writing their own code, they're adding a requirement for the user ("Hey, you gotta have this thing already installed"). The software is in a state of being dependent on each of its dependees, individually. Each such state could be removed by replacing the library, and thus there'd exist one less 'dependency'. ---- Yes, this is mental gymnastics, another fine staple of this glorious field.

by zaarn 5y ago

Except you can't because hundreds of thousands of project have "log4j" in their dependency list not "wyldfire-fixed-log4j". If log4j broke and there was no maintainer, a fork would not fix things.

blibble 5y ago

the documentation also needs to include the 5 line dependency exclusion that you place in your pom to exclude the bad version

zaarn 5y ago
And if you do that everytime an unmaintained project gets a 10 severity CVE, we'll have that at 200 lines in no time. Proabably petition whatever generates your Pom.xml to auto-include those lines.

mantaraygun 5y ago

> hundreds of thousands of project have "log4j" in their dependency list That's on them. They can migrate to wyldfire-fixed-log4j if they want, or to whatever other fork they want, or they can keep using the old busted shit. It's up to them, there's no problem here.

IiydAbITMvJkqKf 5y ago

Sure you can fix it, change the "log4j" in your dependency list to "wyldfire-fixed-log4j". You can't do that with closed source (either because you care about legalities, or because you have no way of obtaining a compilable and readable source code).

zaarn 5y ago
Until all hundreds of thousands of projects have changed that, it'll take time. That doesn't mention the fact you now also have to fork unmaintained projects relying on log4j to use the new version. If you're developing a major java projects, that's potentially thousands of dependencies you have to patch. I'm not suggesting that closed source is the solution, I'm saying the current system is not sustainable and neither is just forking projects. Because which fork do you follow, and how do you make sure all your dependencies follow the fork you want and not an unmaintained fork?
- the_duke 5y ago
  Some package managers (eg yarn , cargo) have a solution for that and allow you to specify a local resolution override that will apply to all dependencies.
  - zaarn 5y ago
    But again, that means EVERY single product, open source or not, will have to patch their deployment to use the fork instead of simply using the updated version of upstream. Permanently no less if it's unmaintained. Meaning every single Java product would have to include that "if you want to use log4j, you have to include this patch line to use a fork, because of unmaintained dependencies". That just ossifies log4j and fixes nothing. Patching dependencies in Cargo is not meant to permanently replace code, it's to be used while you're waiting for an upstream merge.
    - IiydAbITMvJkqKf 5y ago
      You can fix it for your own applications and libraries. The tooling exists. You can't force other people to use your fork.
      - zaarn 5y ago
        Correct, all those other people will have to manually find some fork or make their own, repeating the same work thousands of times and ossifying that some part of your java build system now permanently includes a section to patch log4j dependencies.
        alexvoda 5y ago
        You can not force your dependencies to do what you want. You can only patch the problem for yourself and maintain that patch in the build system, or you can fork each dependency in the chain, maintain those and then your build system is cleaner. There is nothing wrong with these options.
kube-system 5y ago
The problem is, my dependencies also have dependencies. If the problematic dependency is any more than one degree of separation removed, you're back to square one.
- debug-desperado 5y ago
  A nightmare in Node world perhaps (our latest simple web server has over 2,000 dependencies). In the Java world it’s much simpler with a few excludes in your POM. Especially for libraries that keep stable interfaces, such as a logging library.
  - zaarn 5y ago
    How many excludes is Java going to ship is forking the problem of unmaintained libraries away becomes too much? When it reaches 5kb? 10kb? 100kb of excludes?

CuriousCosmic 5y ago

Strictly speaking in the Java space that's a perfectly reasonable assumption. It's a bit more work but you can just substitute the jar/modules for log4j following the build with the patched versions. At the end of the day it is just a bunch of class files in jars/zips which makes patching a lot easier.

zaarn 5y ago
Hot patching a jar is not a scalable solution in the long term, especially if you're considering that in a larger project there may be multiple unmaintained projects that now need permanent hotpatching.
- CuriousCosmic 5y ago
  At least with gradle you can easily add some logic to the buildscript to substitute all instances of a specific dependency for a java library/application with a fork. It's only a handful of lines of code and it is technically a bit hacky but it's really no worse than any of the other jank you are forced to do when building any moderate to complex gradle based java project.
  - zaarn 5y ago
    And when another library is unmaintained and breaks, you add those few lines again. And again and again and again, until your gradle file is 90% hotpatching your dependencies.
    - CuriousCosmic 5y ago
      Yep but that's just the breaks. Especially with ecosystems that have tall dependency trees. Luckily those few lines can mostly be shared and you end up with a handful of lines and a map from old dep to new dep which makes it not too painful. This whole issue is why I don't like these ecosystems but at the very least in the Java space dealing with this issue isn't too painful. I can't speak for the JS space but this is also a pretty trivial fix in the Rust space and it isn't too painful with C++ or C projects. Nix (mostly used for Haskell but it's a pretty solid universal package manager) has some pretty good tooling for modifying upstream dependencies. Spack (package manager specialised for HPC and embedded projects) has really good support for this as well. My point being that people have run into these issues in the past a lot in certain spaces and as a result those spaces have come up with solutions for this problem a long time ago. It just so happens that for the most part the average dev doesn't see a lot of that because these issues get dealt with upstream or just aren't common in most industries. The embedded industry is particularly used to this considering how much of the kit from manufacturers is just outright wrong or broken 3 levels deep. The only reason I'm as well versed with this as I am is because I've been in the embedded space patching over broken HALs and I've bled my blood dealing with legacy enterprise applications that were initially written in Java 1.1 and haphazardly dragged along over the decades. TLDR: This is an ugly fix but ultimately it's the best ugly fix to an ugly problem and every software ecosystem eventually gets to deal with it once they get old and crusty enough.

blindmute 5y ago

Code doesn't just "break". It still works the same. If a vulnerability is discovered you either update the package or switch to a fork. Switching is not really that much extra work compared to updating. This issue has really nothing to do with open source except that you have the extra option of forking.

zaarn 5y ago
I argue that if your code gets assigned a 10/10 Severity CVE, then it broke, even if nothing changed. It just broke a while ago. Switching one library is not much work. But if it becomes the standard approach then the first time you fire up your java project you either spent 30 minutes to put in all the exclusions to provide your favorite forks of dead projects or your pom.xml will already be 5KB large on generation just for those exclusions and patches.

quickthrower2 5y ago

Thats the problem. If you use say React then you are using it as is without any warranty implied. React is the equivalent of an aircraft in complexity when looking at its dependencies and then the browser stack itself on top! React gives you a free copy of an aircraft and will probably maintain it for years. But maybe they rely on an altimeter developed by a poor person who decides they dont have time to work on it because they need to help their mum. Well thats everyone elses problem now. but the ease of download and use of React and other code libraries creates a sense of entitlement. Yes if its broke and you need it fixed you have to pay someone now! Its still libre its just not gratis anymore

dopkew 5y ago

Off-topic but very important: The widespread misuse of the word 'dependency' is very very icky, linguistically speaking. This word originally meant: 'in a state of being dependent'. Usage: "Ronny hated his dependency on the kindness of strangers.". But it's being used to refer to the 'dependent', as well as the 'dependee', which is very poor form: https://english.stackexchange.com/questions/25575/what-is-the-correct-word-for-dependee https://english.stackexchange.com/questions/25575/what-is-th... https://wikidiff.com/dependee/dependent https://wikidiff.com/dependee/dependent

zaarn 5y ago
I don't think that is any issue whatsoever, the software engineering world has largely agreed on the usage of the word dependency, doesn't really matter what linguists think. Language is consensus, not thesis papers.
pritambaral 5y ago
The original meaning still works. In the software world, where abstractions abound, each dependee of yours can be said to add one 'state of being dependent' to your set of requirements. That's what it meant originally in this young field; when an author chooses to use a library instead of writing their own code, they're adding a requirement for the user ("Hey, you gotta have this thing already installed"). The software is in a state of being dependent on each of its dependees, individually. Each such state could be removed by replacing the library, and thus there'd exist one less 'dependency'. ---- Yes, this is mental gymnastics, another fine staple of this glorious field.