Select Page

Introduction

Apache Spark is a distributed open-source system for large-scale data processing and fast general-purpose calculation. It also supports a wide range of advanced tools, including Spark SQL for SQL and DataFrames, pandas API on Spark for pandas workloads, MLlib for machine learning, GraphX for graph processing, and Structured Streaming for stream processing. Its powerful features combined with free access make it widely used worldwide. As a result, the shell command injection vulnerability has a broad impact.

Vulnerability

The shell command injection vulnerability revolves around manipulating application ACLs, which allows users/groups – excluding the owner who starts the application – access to an application. The Apache Spark UI offers the possibility to enable ACLs via the configuration option spark.acls.enable. If ACLs are enabled, Apache spark will go through the HttpSecurityFilter filter to verify the user’s permissions. In the code of HttpSecurityFilter, the username can be entered arbitrarily and the fields will be concatenated later. Therefore, a malicious user might then be able to inject a piece of shell command by separating the characters of the command to several username fields. Then after the fields are concatenated into a piece of the shell command the executeAndGetOutput function will cause the command to execute.

Affected Version

Spark Core – Apache<=3.0.3

3.1.1 <= Spark Core – Apache <=3.1.2

3.2.0 <= Spark Core – Apache <=3.2.1

Remediation

1. Official fix: The vendor has released the patches, and we recommend that users update Apache Spark to 3.1.3, 3.2.2, 3.3.0, or a higher version. Please visit:

https://spark.apache.org/downloads.html

2. Reference methods on GitHub: We can fix the vulnerability by removing the call to bash in the ShellBasedGroupsMappingProvider function. Please visit:

https://github.com/apache/spark/pull/36315/files

Implementing the Fix

The Hillstone IPS can support the detection and protection of this vulnerability.

Figure 1. Apache Spark Shell Command Injection Vulnerability detected by Hillstone IPS

The hotspot intelligence of this vulnerability is available on Hillstone iSource as well.

Figure 2. Hotspot intelligence on Hillstone iSource

Hillstone Security Management V5.3.5 A Super Helper in Your Network

Breaking the Mold: Halting a Hacker’s Code ep. 11 – Kafka Connect JNDI Injection

Treasury Report Calls Out Cybersecurity in the Financial Sector

Hillstone Networks Added as CVE Numbering Authority (CNA)

Hillstone Security Audit V2.19.0: Reigning in Network Complexity with Advanced Log Records

Empowering Cyber Security and Resiliency with Hillstone Networks StoneOS 5.5R10

Breaking the Mold: Halting a Hacker’s Code ep. 10 – Massive ESXiArgs Ransomware Attacks

Breaking the Mold: Halting a Hacker’s Code ep. 9 – XStream Stack Overflow Denial of Service Vulnerability

XDR: A Step Towards Integrated Security for Cyber Defense