Log4j / Log4Shell Vulnerabilities
CVE-2021-45105 | CVE-2021-45046 | CVE-2021-44228 | CVE-2021-42550
Google Translate | | Be aware that Google Translate may change command syntax, as well
1. Executive Summary
Updated: December 30, 2021 @ 3:31 AM CT
The discovery of a new type of attack has opened the door for critical exploits from bad actors and a rush of fixes from Open Source developers to prevent these types of attacks. This attack takes advantage of a series of features in enterprise logging software to call a remote server for sharing configuration across servers. When this feature is instructed to call an LDAP Server, bad actors are able to take advantage of LDAP’s ability to store a specific type of Java data and deliver an exploit to the source system.
After the initial exploit was remediated, several other attacks were uncovered by security researchers in the widely used Log4j2 and Logback software libraries. These exploits impact Java-based software which is widely used in the enterprise and Internet technology companies.1.1 What to expect
When any new attack vector is discovered, additional exploits will be discovered across other applications and open source frameworks. Technical leaders should prepare for more exploits in the coming months.
1.2 Course of action
Technical leaders should ensure they have visibility to the software running in their environment and the ability to verify the version number of libraries and components. As new exploits are reported, technical teams are then well positioned to know which systems need to be upgraded and to immediately begin upgrading to remediate the security risk.
2. Background
A zero-day vulnerability was disclosed in the world’s most widely adopted logging framework, Log4j. Within 24 hours of the disclosure, global organizations were reporting exploits. Apache has identified the issue and published a fix.
This is a commonly used package and the impact of the vulnerability has been covered by many on the Internet. HYTE Support would like to provide some resources regarding this vulnerability in the event that you haven’t seen this information yet.
The issue is readily mitigated from exploitation with a simple configuration property. HYTE is working with Open Source communities to release updates that include the patch moving forward.
The Apache Log4j project has released a new version (v2.17.0) which requires that users explicitly enable the problematic lookups. This version will be used in the HYTE releases moving forward starting with v5.2.30 and v5.4.0.
Table 1. CVE Matrix
CVE Number | Component | Severity | Score |
---|---|---|---|
CVE-2021-42550 | LogBack | High | 8.5 |
CVE-2021-44228 | Log4j2 | Critical | 10.0 |
CVE-2021-44832 | Log4j2 | Moderate | 6.6 |
CVE-2021-45046 | Log4j2 | Critical | 9.0 |
CVE-2021-45105 | Log4j2 | Moderate | 5.9 |
3. Exposure
Log4j2 is massively popular and widely used. Virtually every enterprise and Internet technology company will be impacted directly or indirectly.
This popularity is seen as a massive opportunity for bad actors. Reports from Sonatype, (which operates the most popular Java library repository in the world-- Maven Central) have reported the Log4j2 was downloaded over 28 million times in the last 4 months and is used by almost 7,000 open source projects-- including the helicopter launched by the Mars rover (Ingenuity). The feature that is being exploited is also very popular. Almost 800 other projects borrow the problematic JndiManager class from Log4j2 for use in their system.4. Exploits
Unsavory actors are actively exploiting this vulnerability with identified hosts in Russia and other hosts running on public Internet addresses.
4.1 Blocking public Internet access
Environments that do not permit their enterprise systems from accessing public Internet are at reduced risk for immediate exploitation.
4.2 Next wave of attacks
Computer security experts are alerting that the next wave of attacks will include chaining other vulnerabilities to exploit environments, including ransomware attacks.
Blocking public Internet access alone will not be sufficient in preventing future attacks. Fixes need to be applied in order to mitigate attacks where the Log4Shell vulnerability is combined with other security exploits to compromise systems
✕5. Resolution
The fundamental nature of the logging software being the affected party means that it will require some time to ensure proper compatibility and stability of all layers throughout HYTE products.
HYTE will be providing customers with resolution in three phases Phase 1 - Mitigation solution available immediately Customers are encouraged to apply the appropriate mitigation solution to all instances in all environments. Phase 2 - Mitigation solution configured by default HYTE Platform v5.2.28 will include the mitigation flag enabled by default. Customers may choose to upgrade their environments to this version. Phase 3 - Permanent patches applied in all releases HYTE Platform v5.2.30 and upcoming v5.4.x releases will include the permanent patches from upstream Open Source communities.6. Mitigation Technical Details
The Apache Log4j2 community responded quickly with updates to address the vulnerabilities. After additional exploits were discovered, the problematic feature was removed entirely in v2.16.0.
6.1 Apache Log4j2 mitigation matrix
Table 2. Apache Log4j2 version Mitigation
Log4j Version | Mitigation |
---|---|
2.17.1 | Fourth release, addresses moderate risk flaw |
2.17.0 | Third release, fixes Critical and High risk flaws |
2.16.0 | Second release in response to Log4Shell, improvements |
2.15.0 | First release in response to Log4Shell, Option 1 set by default |
>= 2.10 | Option 3 |
2.7 =< version used =< 2.14.1 | Option 3 |
2.0-beta9 =< version used =< 2.10.0 | Option 3 |
6.2 Apache ActiveMQ mitigation matrix
Apache ActiveMQ distributions are not impacted by the Log4Shell vulnerability, as all the current releases use Log4j v1.x
The upcoming Apache ActiveMQ 5.17.0 release includes an upgrade to Log4j2, and the upstream source tree has been patched to the fixed version of Log4j2 (v2.17.1).Table 3. Apache ActiveMQ version Mitigation
Apache ActiveMQ Version | Mitigation |
---|---|
5.17.0 (unreleased) | Fixed. Upstream source patched to Log4j2 2.17.1 |
5.x (all versions) | Not vulnerable |
6.3 HYTE Product mitigation matrix
HYTE products that use the HYTE Runtime are powered by Apache Karaf, which in turn utilizes Log4j2 for features and performance benefits. HYTE does not use LogBack.
- HYTE Console
- HYTE MQ
- HYTE Runtime
Table 4. HYTE Product Matrix
HYTE Product Version | HYTE OSS Version | Log4j Version | Mitigation |
---|---|---|---|
5.2.28 | 4.2.11.hyte-42113 | 2.14.0 | Option 2 + 3 (enabled by default) |
5.2.x | 4.2.11.hyte+ | 2.14.0 | Option 2 + 3 |
5.0.10 | 4.2.10.hyte-42101 | 2.13.3 | Option 2 + 3 |
N/A | 4.2.5.hyte-4200+ | 2.8.2 | Option 2 + 3 |
N/A | 4.1.5.hyte-4005+ | 2.8.2 | Option 2 + 3 |
N/A | Older | Varies | Option 2 + 3 |
7. Mitigation Options
Table 5. Mitigations
Mitigation | Approach | Effective | Notes |
---|---|---|---|
Option 1 | Disable lookup feature | No | Not effective |
Option 2 | Disable lookups in log pattern | No | Must be combined with Option 3 |
Option 3 | Remove problematic class | Yes | |
Option 4 | Upgrade to Log4j2 2.17.0 | Yes |
7.1 Option 1: Disable message format lookups
Option 1 is ineffective✕
Option 1 is ineffective
✕7.2 Option 2: Update logging message format
Must be combined with Option 3✕
Must be combined with Option 3
✕This change disables message format lookups for individual log configurations.
Configuration macro:{nolookups}The vulnerability is mitigated from exploitation by updating logging configuration file:
etc/org.ops4j.pax.logging.cfgBefore:
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%nAfter
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m{nolookups}%n
This needs to be changed for every pattern used
✕
7.3 Option 3: Remove JndiLoookup class
This removes the class from the installation to prevent exploitation.
This does not effect logging moving forward, as Log4j2 quietly skips lookups when this class is missing. Steps: 1. Identify the Log4j bundle idkaraf@root()> list -t 0 | grep -i log4j 7 │ Active │ 8 │ 1.11.9 │ OPS4J Pax Logging - Log4Jv2 implementation, Fragments: 14 14 │ Resolved │ 12 │ 1.11.9 │ OPS4J Pax Logging - Log4j v2 Extra packages, Hosts: 7 karaf@root()>The log4j2 bundle is '7' in this example 2. Confirm the JndiLookup class exists
karaf@root()> classes 7 | grep -i jndilookup org/apache/logging/log4j/core/lookup/JndiLookup.class | exported: false karaf@root()>3. Stop the process
./bin/stop4. Remove the class from the system/ folder
$ zip -q -d system/org/ops4j/pax/logging/pax-logging-log4j2/1.11.9/pax-logging-log4j2-1.11.9.jar org/apache/logging/log4j/core/lookup/JndiLookup.class5. Remove the class from the deployed bundle in the data/cache folder
$ zip -q -d data/cache/bundle7/version0.0/bundle.jar org/apache/logging/log4j/core/lookup/JndiLookup.class6. Start the process
./bin/start7. Confirm the class is removed
karaf@root()> classes 7 | grep -i jndilookup karaf@root()>
7.4 Option 4: Upgrade Log4j2 to 2.17.1
As of 2021-12-21, the Log4j2 2.17.0 release fixes all known security issues.
HYTE is planning on releasing two updates with fixes built-in. This approach provides a buffer from the large changeset introduced in Log4j2 2.16.0 and 2.17.0, (which was understandably rushed to release). Table 6. HYTE Release MatrixHYTE Product Version | HYTE OSS Version | Log4j Version | Mitigation |
---|---|---|---|
5.2.28 | 4.2.11.hyte-42113e | 2.14.0 | Option 2 + 3 (enabled by default) |
5.2.30 | 4.2.14.hyte-42140 | 2.17.0 | Option 4 |
5.4.0 | 4.3.5.hyte-43050 | 2.17.0 | Option 4 |
8. Ineffective Options
HYTE performed internal testing to validate and recommend mitigation options. The following options have been proposed by various parties across various forums on the Internet. These options have not been able to be proven by HYTE Engineering, or incur significant security risk.
HYTE recommends customers do NOT rely on any of the following options
✕8.1 Option 1: Disable message format lookups
Option 1 mitigation is no longer sufficient✕
This change disables message format lookups for the entire process.
The vulnerability is mitigated from exploitation by setting a Java system property at boot time:
Option 1 mitigation is no longer sufficient
✕-Dlog4j2.formatMsgNoLookups=trueSteps: 1. Update the bin/setenv file:
export LOG4SHELL_OPTS="-Dlog4j2.formatMsgNoLookups=true" export EXTRA_JAVA_OPTS="$JAVA_G1_GC_OPTS $JAVA_GC_LOG_OPTS $JAVA_SP_LOG_OPTS $JAVA_HEAPDUMP_OPTS $JAVA_NET_OPTS -Dlog4j2.formatMsgNoLookups=true"2. Restart the HYTE process
./bin/stop ./bin/start3. Verify the property is set
$ ps -aef | grep -i hyteConfirm property is present:
hyte 2429254 2429192 99 18:24 pts/0 00:00:21 /opt/hyte/java/active/bin/java ... -Dlog4j2.formatMsgNoLookups=true ... org.apache.karaf.main.Main
8.2 Java JDK flags
Early reports indicated that making configuration changes to the Java JDK would mitigate the flaw. HYTE’s internal testing has NOT been able to successfully mitigate the flaw using these options.
1. Ineffective configuration properties-Dcom.sun.jndi.ldap.object.trustURLCodebase=false -Dcom.sun.jndi.rmi.object.trustURLCodebase=false -Dcom.sun.jndi.cosnaming.object.trustURLCodebase=false
8.3 Flaw specific Java Agents
Several prominent technical influencers have produced Java Agents and other alternate tools to address the flaw. HYTE does NOT recommend customers apply these approaches for the following reasons:
1. Risk of bringing in Trojan horse or other exploit payload masked as a 'fix' 2. This is brand new code and requires additional operation steps to uninstall once the permanent fixes are available 3. Potentially introduce new bugs or security flaws9. Frequently Asked Questions
What is Log4j?
Log4j (or Log4j2) is the most popular logging library for applications written in Java.
What is LogBack?
Logback is a very popular alternative logging library for applications written in Java.
Why does HYTE use Log4j2?
Log4j2 provides very high performance logging without dropping log messages under heavy load. This is critical for high-performing messaging environments.
https://logging.apache.org/log4j/2.x/performance.html https://www.loggly.com/blog/benchmarking-java-logging-frameworks/10. Change Log
Table 7. Changes
Date | Note |
---|---|
2021-12-29 | Updated for Log4j2 2.17.1 |
2021-12-21 | Updated to account for latest updates |
2021-12-13 | Updated mitigation matrix |
2021-12-12 | JDK upgrade secondary mitigation option questioned |
2021-12-11 | JDK upgrade secondary mitigation option communicated |
2021-12-10 | Initial publication |