A vulnerability dubbed HollowByte allows unauthenticated attackers to trigger a denial-of-service (DoS) condition on OpenSSL servers with a malicious payload of just 11 bytes.
The OpenSSL team has silently fixed the vulnerability (no identifier assigned) and backported the patch to older releases.
Because the OpenSSL software is the foundational backbone for secure internet communication, organizations should prioritize switching to a fixed version of the library.
In an advisory earlier this week, Okta’s Red Team described how the HollowByte DoS vulnerability works and its impact in a real-world scenario.
The researchers explain that in a TLS handshake, each message has a 4-byte header for declaring the size of the incoming message. However, vulnerable OpenSSL versions allocate the declared length before receiving the payload and checking its size.
Every TLS handshake message begins with a 4-byte handshake header, where a three-byte length field discloses the size of the handshake data that should follow.
Without validating the payload, the server trusts the packet’s claims and allocates the indicated memory. “The worker thread then blocks, waiting indefinitely for data that will never arrive,” Okta explains.
An unauthenticated attacker can trigger HollowByte by opening a TLS connection and sending an 11-byte malicious input with a header declaring that a much larger message body will follow.
The attacker repeats the same process across multiple connections, causing the server to allocate considerable amounts of memory via a relatively small volume of transmitted data.
Okta researchers note that while OpenSSL frees the buffers when a connection drops, the GNU C Library (glibc) has a different way to handle memory and “does not immediately return small-to-medium allocations to the operating system; it keeps them for potential reuse.”
“By launching waves of connections with randomized claimed sizes, an attacker prevents the allocator from reusing those freed chunks,” Okta says.
“The heap fragments heavily, causing the server’s Resident Set Size (RSS) to climb continuously. Even after the attacker disconnects, the server remains permanently bloated.”
The only way to fully reclaim the space is by restarting the process.
The open-source OpenSSL library is embedded in popular software projects such as NGINX and Apache web servers, language runtimes (e.g., Node.js, Python, Ruby, PHP), and databases (MySQL, PostgreSQL). It comes pre-installed on most Linux distributions for TLS encryption and certificate handling.
In Okta’s tests on NGINX showed that low-capacity environments can be easily depleted of memory using HollowByte, while higher-spec servers may lose up to 25% of their memory while the attack bandwidth remains below security alerting thresholds.
Although DoS flaws are considered less severe than vulnerabilities that enable data theft or code execution, they can cause operational disruptions and reputational damage.
The HollowByte DoS issue has been fixed in OpenSSL 4.0.1 and backported to versions 3.6.3, 3.5.7, 3.4.6, and 3.0.21, which now grow the buffer only when the data arrives, ignoring header claims.
Despite being addressed as a “hardening fix” and not a security vulnerability, Okta recommends “upgrading your distribution’s OpenSSL packages immediately.”
Security teams log 54% of successful attacks and alert on just 14%. The rest move through your environment unseen.
The Picus whitepaper shows how breach and attack simulation tests your SIEM and EDR rules so threats stop slipping by detection.
FFmpeg fixes PixelSmash flaw in widely used video decoder
F5 issues out-of-band patches for critical NGINX vulnerabilities
phpBB forum fixes auth bypass bug lurking for a decade
New ‘HTTP/2 Bomb’ DoS attack crashes web servers in under a minute
CISA: Hackers now exploit SolarWinds Serv-U flaw to crash servers
Ernst & Young discloses data breach after support system hack
New Windows LegacyHive zero-day gives hackers admin privileges
CISA urges immediate action on actively exploited Fortinet flaws
AI is a data-breach time bomb: Read the new report
AI broke vulnerability management. Get the guide CISOs use to shift budget to BAS.
Privacy by policy or privacy by architecture? See how age checks work when the face never leaves the device.
See how Pixellot discovered and secured hundreds of unmanaged AI agent identities in weeks, not months. Read the case study.
Pentest your web apps on-demand. Find what humans miss. Scope and launch a pentest in minutes.
Do you have what it takes to challenge DARKROOM? Signup for an exclusive DEFCON CTF!
Terms of Use – Privacy Policy – Ethics Statement – Affiliate Disclosure
Read our posting guidelinese to learn what content is prohibited.
