11 ms·

Ideally, log4j should not have the ability to make "outgoing calls" right? It has been years since I used it but I can't remember a valid reason why it should

Ideally, log4j should not have the ability to make "outgoing calls" right? It has been years since I used it but I can't remember a valid reason why it should be allowed to do so. Since it is running inside the stack it may not be easy to enforce it. If the "client" log4j does it before even logging that is a bother. It seems like having a "central" "syslog" logging server. Traffic goes from the stack -> logging server. Logging server not allowed to call anyone external.

bob1029 5y ago
> Ideally, log4j should not have the ability to make "outgoing calls" right? What is the intended scope of log4j? I spent some time looking over the source code and cannot fathom why something that logs information would need to become this complicated.
- throwaway894345 5y ago
  Welcome to the Java ecosystem. Please enjoy your stay.
- mjr00 5y ago
  > I spent some time looking over the source code and cannot fathom why something that logs information would need to become this complicated. That's easy: because software development is complicated. If you look at the ticket where this was added[1], it seems like a reasonable request for something a very featureful logging library might do. Other logging frameworks[2] already supported this feature. It's easy to armchair quarterback and say the feature shouldn't have been added, but the reality is an old project like this that's used by many large enterprises with specific requirements is going to naturally bloat as it ages. Anyone who's been in software dev long enough has experienced this. You build a super-simple internal tool or library that does solves a specific problem you have. It starts getting used. Then you get a feature request from Department A to do something slightly different. Department B says you need to integrate the tool with their new CRM platform. The security team gets involved and says you need to integrate auditing of your tool's usage into their metrics. etc... For Apache/log4j, it's not just multiple departments, but a wide array of companies across the entire world. It's very easy to explain feature bloat. [1] https://issues.apache.org/jira/browse/LOG4J2-313 https://issues.apache.org/jira/browse/LOG4J2-313 [2] http://logback.qos.ch/manual/contextSelector.html http://logback.qos.ch/manual/contextSelector.html
  - svieira 5y ago
    Yet the exact same kind of feature for Logback requires _explicit opt-in_ on the part of the application developer, via a system property (`logback.ContextSelector=JNDI`) that _must be set before Logback is first instantiated_. If you want custom behavior then you, the application developer must write extra configuration or even code in order to wire Logback into your system correctly. Logback doesn't attempt to make this work for everyone for everything. It's a library. Log4J implemented the same thing and tried to make it Just Work for everyone. Unfortunately, that "everyone" includes people who are not even legitimate end-users of the application. Make the hard cases "possible", don't try to make them "easy" (especially not without several solid use-cases around "hard-but-should-be-easy".)
    - mjr00 5y ago
      Oh yeah don't get me wrong, the implementation wasn't great. It should have been opt-in at a minimum, and in some kind of separate extension ideally. But there's been so much incredulity in these comment sections about the log4j issue, as if it were totally obvious that the feature was inherently bad and should have never been in an enterprise logging framework, which I don't think is true.
- emaginniss 5y ago
  All software projects must grow in complexity until they include a DSL. Or make outgoing arbitrary network calls.
- formerly_proven 5y ago
  fwiw Python's logging module is a port of a very old log4j version (think 2000 vintage) and is already too complicated with unnecessary abstractions and features, yet is also physically incapable of commonly needed features (like scoping logging, buffering logs for errors, selecting which formatting type to use etc.) ... like many Python libraries from that time period when top Python brass was imitating Java heavily, it's really quite bad but also "the standard".
staticassertion 5y ago
That would help but it isn't enough. You can still load classes from your class path, and then those would make the calls instead. And log4j does have valid reasons to read/write to files, naturally. But yes, I think a major issue here is a logging library packaging its own meta language that can trigger side effects.
layer8 5y ago
The main design mistake in log4j is that it performs ${…} substitution on the complete log message by default. There’s no way for application code using the log4j front-end API to have input data containing such strings be logged verbatim, without nonstandard log4j back-end configuration (someone correct me if there’s actually a way). In principle, each application has to sanitize all strings to be logged by stripping/escaping all log4j substitution syntax from them, to prevent unexpected substitutions.
- Sohcahtoa82 5y ago
  Why did the substitution syntax ever even allow JNDI calls? I mean seriously, who thought this was a good idea? It's about as dumb as allowing a SQL server to execute shell commands or read/write directly to files, or for XML documents to allow external entities that can fetch from URLs or local files. Even worse is when these gaping security holes are enabled by default.
  - layer8 5y ago
    See the description here for the rationale why JNDI support was added: https://issues.apache.org/jira/plugins/servlet/mobile#issue/LOG4J2-313 https://issues.apache.org/jira/plugins/servlet/mobile#issue/... Nobody there seemed to be aware that the log message text itself is also subject to lookup syntax interpretation and that this would be dangerous given that JNDI also allows remote code-base download and execution from the specified URL.
    - Fabricio20 5y ago
      Except this particular issue has nothing to do with LDAP! It's perfectly reproducible with RMI for example, another JNDI supported protocol. So much so that the initial RC1 patch was an IF for LDAP, and it was obviously bypassed in seconds.
      - layer8 5y ago
        You are right, I edited my comment now.
kerblang 5y ago
Apparently the intention was to add the ability to parse random crap one might wish to put in config files, including things like pulling a string out of a JNDI reference, system variables (maybe environment variables?); the person making the change did not understand that the scope extended into actual log printing as well. I probably spend more time grepping dependencies than googling for fixes. Scope kills.
throwaway894345 5y ago
Personally I'd like to see more runtimes with some concept of capabilities. As a user, I'd like to be able to say "this package cannot make outbound calls" rather than doing so at the program or host grain via layer 3 or layer 7 firewall rules. I think maybe Deno allows this, and I wouldn't be surprised if JVM or .Net did as well. Failing that, it would be interesting to see this implemented as a linter--low level I/O and HTTP functions are annotated such that callers must have the appropriate capability annotation in order to call them. This wouldn't prevent a nefarious library publisher from making HTTP requests through an FFI or an alternate HTTP library unknown to the linter, but it would prevent attackers from exploiting bugs/misfeatures in libraries as in this log4j case.
- layer8 5y ago
  That’s pretty much what the Security Manager is for, which Java has had since the 1.0 version. However it was rarely applied outside of Java applets, and is scheduled to be removed [0] because it is difficult to apply effectively and doesn’t cover all types of vulnerabilities. [0] https://openjdk.java.net/jeps/411 https://openjdk.java.net/jeps/411
- throwaway19937 5y ago
  > Personally I'd like to see more runtimes with some concept of capabilities. As a user, I'd like to be able to say "this package cannot make outbound calls" rather than doing so at the program or host grain via layer 3 or layer 7 firewall rules. I think maybe Deno allows this, and I wouldn't be surprised if JVM or .Net did as well. Unfortunately both of these platforms have removed their security controls - the JVM's SecurityManager[0] was deprecated and recent versions of .NET have dropped sandboxing. [0] https://openjdk.java.net/jeps/411 https://openjdk.java.net/jeps/411
- foobiekr 5y ago
  You should do it on the network _anyway_. You want multiple layers because you often find that any given layer was not as robust as you assumed.
  - deleted 5y ago
rodgerd 5y ago
> Ideally, log4j should not have the ability to make "outgoing calls" right? If it's in the application execution context, it will have the same abilities. If your middleware needs to make calls to third-parties, and it uses log4j, the log4j code has the same rights as the middleware app. If you're using a registry like Nexus, and it uses log4j, then it also has the ability to make calls to Maven repos, pypi repos, and anything else that Nexus can reach. And so on and so forth. Essentially you're expecting there to be privilege separation, where there is none.
foobiekr 5y ago
Suppose that even if it needs this for some kind of log transformation pipeline, there's still absolutely no reason for the jvm process in which this code is running to be able to reach out to some random internet location. Do people just not put strict segmentation in place? If so, for the love of god, why? It's not that hard and it is a very useful baseline.

bob1029 5y ago

> Ideally, log4j should not have the ability to make "outgoing calls" right? What is the intended scope of log4j? I spent some time looking over the source code and cannot fathom why something that logs information would need to become this complicated.

throwaway894345 5y ago
Welcome to the Java ecosystem. Please enjoy your stay.
mjr00 5y ago
> I spent some time looking over the source code and cannot fathom why something that logs information would need to become this complicated. That's easy: because software development is complicated. If you look at the ticket where this was added[1], it seems like a reasonable request for something a very featureful logging library might do. Other logging frameworks[2] already supported this feature. It's easy to armchair quarterback and say the feature shouldn't have been added, but the reality is an old project like this that's used by many large enterprises with specific requirements is going to naturally bloat as it ages. Anyone who's been in software dev long enough has experienced this. You build a super-simple internal tool or library that does solves a specific problem you have. It starts getting used. Then you get a feature request from Department A to do something slightly different. Department B says you need to integrate the tool with their new CRM platform. The security team gets involved and says you need to integrate auditing of your tool's usage into their metrics. etc... For Apache/log4j, it's not just multiple departments, but a wide array of companies across the entire world. It's very easy to explain feature bloat. [1] https://issues.apache.org/jira/browse/LOG4J2-313 https://issues.apache.org/jira/browse/LOG4J2-313 [2] http://logback.qos.ch/manual/contextSelector.html http://logback.qos.ch/manual/contextSelector.html
- svieira 5y ago
  Yet the exact same kind of feature for Logback requires _explicit opt-in_ on the part of the application developer, via a system property (`logback.ContextSelector=JNDI`) that _must be set before Logback is first instantiated_. If you want custom behavior then you, the application developer must write extra configuration or even code in order to wire Logback into your system correctly. Logback doesn't attempt to make this work for everyone for everything. It's a library. Log4J implemented the same thing and tried to make it Just Work for everyone. Unfortunately, that "everyone" includes people who are not even legitimate end-users of the application. Make the hard cases "possible", don't try to make them "easy" (especially not without several solid use-cases around "hard-but-should-be-easy".)
  - mjr00 5y ago
    Oh yeah don't get me wrong, the implementation wasn't great. It should have been opt-in at a minimum, and in some kind of separate extension ideally. But there's been so much incredulity in these comment sections about the log4j issue, as if it were totally obvious that the feature was inherently bad and should have never been in an enterprise logging framework, which I don't think is true.
emaginniss 5y ago
All software projects must grow in complexity until they include a DSL. Or make outgoing arbitrary network calls.
formerly_proven 5y ago
fwiw Python's logging module is a port of a very old log4j version (think 2000 vintage) and is already too complicated with unnecessary abstractions and features, yet is also physically incapable of commonly needed features (like scoping logging, buffering logs for errors, selecting which formatting type to use etc.) ... like many Python libraries from that time period when top Python brass was imitating Java heavily, it's really quite bad but also "the standard".

staticassertion 5y ago

That would help but it isn't enough. You can still load classes from your class path, and then those would make the calls instead. And log4j does have valid reasons to read/write to files, naturally. But yes, I think a major issue here is a logging library packaging its own meta language that can trigger side effects.

layer8 5y ago

The main design mistake in log4j is that it performs ${…} substitution on the complete log message by default. There’s no way for application code using the log4j front-end API to have input data containing such strings be logged verbatim, without nonstandard log4j back-end configuration (someone correct me if there’s actually a way). In principle, each application has to sanitize all strings to be logged by stripping/escaping all log4j substitution syntax from them, to prevent unexpected substitutions.

Sohcahtoa82 5y ago
Why did the substitution syntax ever even allow JNDI calls? I mean seriously, who thought this was a good idea? It's about as dumb as allowing a SQL server to execute shell commands or read/write directly to files, or for XML documents to allow external entities that can fetch from URLs or local files. Even worse is when these gaping security holes are enabled by default.
- layer8 5y ago
  See the description here for the rationale why JNDI support was added: https://issues.apache.org/jira/plugins/servlet/mobile#issue/LOG4J2-313 https://issues.apache.org/jira/plugins/servlet/mobile#issue/... Nobody there seemed to be aware that the log message text itself is also subject to lookup syntax interpretation and that this would be dangerous given that JNDI also allows remote code-base download and execution from the specified URL.
  - Fabricio20 5y ago
    Except this particular issue has nothing to do with LDAP! It's perfectly reproducible with RMI for example, another JNDI supported protocol. So much so that the initial RC1 patch was an IF for LDAP, and it was obviously bypassed in seconds.
    - layer8 5y ago
      You are right, I edited my comment now.

kerblang 5y ago

Apparently the intention was to add the ability to parse random crap one might wish to put in config files, including things like pulling a string out of a JNDI reference, system variables (maybe environment variables?); the person making the change did not understand that the scope extended into actual log printing as well. I probably spend more time grepping dependencies than googling for fixes. Scope kills.

throwaway894345 5y ago

Personally I'd like to see more runtimes with some concept of capabilities. As a user, I'd like to be able to say "this package cannot make outbound calls" rather than doing so at the program or host grain via layer 3 or layer 7 firewall rules. I think maybe Deno allows this, and I wouldn't be surprised if JVM or .Net did as well. Failing that, it would be interesting to see this implemented as a linter--low level I/O and HTTP functions are annotated such that callers must have the appropriate capability annotation in order to call them. This wouldn't prevent a nefarious library publisher from making HTTP requests through an FFI or an alternate HTTP library unknown to the linter, but it would prevent attackers from exploiting bugs/misfeatures in libraries as in this log4j case.

layer8 5y ago
That’s pretty much what the Security Manager is for, which Java has had since the 1.0 version. However it was rarely applied outside of Java applets, and is scheduled to be removed [0] because it is difficult to apply effectively and doesn’t cover all types of vulnerabilities. [0] https://openjdk.java.net/jeps/411 https://openjdk.java.net/jeps/411
throwaway19937 5y ago
> Personally I'd like to see more runtimes with some concept of capabilities. As a user, I'd like to be able to say "this package cannot make outbound calls" rather than doing so at the program or host grain via layer 3 or layer 7 firewall rules. I think maybe Deno allows this, and I wouldn't be surprised if JVM or .Net did as well. Unfortunately both of these platforms have removed their security controls - the JVM's SecurityManager[0] was deprecated and recent versions of .NET have dropped sandboxing. [0] https://openjdk.java.net/jeps/411 https://openjdk.java.net/jeps/411
foobiekr 5y ago
You should do it on the network _anyway_. You want multiple layers because you often find that any given layer was not as robust as you assumed.
- deleted 5y ago

rodgerd 5y ago

> Ideally, log4j should not have the ability to make "outgoing calls" right? If it's in the application execution context, it will have the same abilities. If your middleware needs to make calls to third-parties, and it uses log4j, the log4j code has the same rights as the middleware app. If you're using a registry like Nexus, and it uses log4j, then it also has the ability to make calls to Maven repos, pypi repos, and anything else that Nexus can reach. And so on and so forth. Essentially you're expecting there to be privilege separation, where there is none.

foobiekr 5y ago

Suppose that even if it needs this for some kind of log transformation pipeline, there's still absolutely no reason for the jvm process in which this code is running to be able to reach out to some random internet location. Do people just not put strict segmentation in place? If so, for the love of god, why? It's not that hard and it is a very useful baseline.