Two years ago, we launched the Extended Security Maintenance (ESM) phase of Ubuntu 14.04, providing access to CVE patches through an Ubuntu Advantage for Infrastructure free or paid subscription. This phase extended the lifecycle of Ubuntu 14.04 LTS, released in April 2014, from the standard, five years of an LTS release to a total of eight years, ending in April 2022. During the ESM phase we release security fixes for high and critical priority vulnerabilities for the most commonly used packages in the Ubuntu main and restricted archives. In this post, I would like to review and share our experience from the past two years of maintaining this release
To date, in the lifecycle of Ubuntu 14.04 ESM we published 238 Ubuntu Security Notices (USN), covering 574 CVEs ranging from high-low in priority. The ensuing security updates, protected from vulnerabilities with impacts ranging from remote code execution and privilege escalation, to CPU hardware vulnerabilities. Our average time of resolving high-priority CVEs, was 14 days.
The software vulnerabilities that stood out
Not all vulnerabilities are the same, and high publicity, or a lavish name, does not always imply high risk. Below I have unpacked the vulnerabilities which had the most impact on the 14.04 ESM lifecycle -SACK Panic, GRUB2 BootHole, Baron Samedit and Exim remote code execution.
SACK Panic – kernel
The Linux kernel, being a core part of any system’s attack surface, has the potential for high impact vulnerabilities. What can be worse? A vulnerability on its TCP stack that can be triggered remotely. SACK Panic is a set of Denial of Service (DoS) vulnerabilities that apply to both server and client systems and can be remotely triggered. It takes advantage of an integer overflow and other flaws in the Linux kernel’s TCP implementation of Selective Acknowledgement (SACK). We fixed it with USN-4017-2 in June 2019
GRUB2 BootHole – GRUB2
The UEFI secure boot establishes the integrity of the operating system starting from the UEFI BIOS, and authenticating the subsequent software involved during boot. In our case that is GRUB2, which is in turn authenticating the Linux kernel. That process prevents malware from becoming persistent by getting outside the operating system’s boundary. GRUB2 contained various vulnerabilities, including integer overflow with heap-buffer overflow and use-after free, that would allow its authentication part to be bypassed. A local attacker with administrative privileges or with physical access to the system could circumvent GRUB2 module signature checking, resulting in the ability to load arbitrary GRUB2 modules that have not been signed by a trusted authority and hence bypass UEFI Secure Boot. It was fixed with USN-4432-1 in July 2020.
Baron Samedit – sudo
Sudo is one of the most common tools used by administrators. It allows performing privileged tasks while running as an unprivileged user, providing not only safety by allowing to operate unprivileged by default, but also creates an audit trail of who performed what in an environment with multiple administrators. This vulnerability can cause an unauthorized privilege escalation by local users, due to incorrect memory handling in sudo (heap buffer-overflow), present in the codebase since 2011. We addressed it with USN-4705-2 in January 2021.
Exim remote code execution vulnerability
Exim is one of the main mail (SMTP) handling servers in Ubuntu. This vulnerability can result in remote command execution due to a mismatch between the encoder and the decoder of a part of the SMTP protocol. It was fixed with USN-4124-1 in September 2019.
The hardware vulnerabilities that stood out
Hardware, unlike software, cannot easily be updated or patched in the field. As such, when vulnerabilities are found within the hardware itself, it is the software on top of the hardware that must work around the issue. Hardware has a very limited degree of adjustment, e.g., via the drivers or with microcode. Here I recap the most prominent hardware vulnerabilities that we addressed during the first two years of the 14.04 ESM lifetime
Intel Microarchitectural Data Sample (MDS) vulnerabilities
This vulnerability on Intel CPUs causes a confidentiality breach. Data used by applications running on the same CPU may be exposed to a malicious process that is executing on the same CPU core. This is a complex attack which uses a speculative execution side-channel, and cannot be easily used for targeted attacks (see our wiki for the more detailed review). This vulnerability is mitigated with kernel, qemu, libvirt and microcode updates released with USN-3977-1, USN-3977-3, USN-3983-1, USN-3982-2, USN-3981-2, USN-3985-2, USN-3977-2, USN-3978-1 in May 2019.
The TSX Asynchronous Abort
The TSX Asynchronous Abort (TAA) vulnerability allows an attacker to access the CPU’s microarchitectural buffers through the Intel® Transactional Synchronization Extensions (Intel® TSX). This is again a data confidentiality vulnerability, that can cause memory exposure between userspace processes, between the kernel and userspace, between virtual machines, or between a virtual machine and the host environment. The issue is mitigated by a combination of updated processor microcode and Linux kernel updates, released with USN-4187-1, USN-4186-2 in November 2019
Intel graphics (i915) vulnerabilities
These are two vulnerabilities that affect Intel Graphics processor. The first (CVE-2019-0154) can cause a local user triggered Denial of Service (DoS) attack. That is, a system hang can be caused by an unprivileged user performing a read from GT memory mapped input output (MMIO) when the processor is in certain low power states
The second vulnerability (CVE-2019-0155) is a privilege escalation and a breach of data confidentiality. The graphics processors allowed unprivileged users to write and read kernel memory, resulting in possible privilege escalation.
Both vulnerabilities are mitigated by updates to the Intel graphics driver as part of Linux kernel updates, released with USN-4187-1, USN-4186-2 in November 2019.
Taking another look
The most impactful software vulnerabilities solved in 14.04 ESM took advantage of well known techniques such as heap-buffer overflows, use-after-free, integer overflows as well as logic errors. While there are mitigations and defenses in play such as Non-Executable Memory available in modern CPUs, as well as the heap-protector, ASLR and others in Ubuntu 14.04, these raise the bar for the attacker, but do not provide complete protection, and attacks only get better. Hence regular vulnerability patching is a key element in reducing the risk of a breach.
Taking a step back on the hardware vulnerabilities, we see that CPUs are no longer seen as black boxes, and we see a steady stream of vulnerabilities identified and mitigated. In the presence of these vulnerabilities, shared hosting, e.g., on a cloud, cannot always guarantee the confidentiality of data processed by the shared CPUs. It is imperative to note that the necessary privacy on the cloud comes not only from the hardware and operating system integration, but also requires a well maintained system with regular vulnerability patching in place.
The packages with most security vulnerabilities
Although there are more than 2000 packages in Ubuntu 14.04 ESM main repositories, there were packages that stood out in our vulnerability fixes. This does not imply a design weakness, but can also be attributed to their popularity and importance to the open source ecosystem that attracts researchers’ attention. Our graph below displays the number of security fixes in the top packages that we patched.
To upgrade or not upgrade?
Transitioning to the latest operating system is important for performance, hardware enablement, software fixes and technology enablement benefits. However, upgrading is not a simple, quick or inexpensive process. Enterprise solutions combine software from a variety of teams within an organization, and in most cases there is an extended supply chain, involving software from 3rd party vendors, who in turn may have their own software vendors.
Such complex scenarios result in a dependency on software stacks (e.g., Java, python) that have certain properties in the upgraded system that either got deprecated, replaced or slightly changed behavior in the newer system. The upgrade process in that case becomes a change management process involving risk analysis, stakeholder communication and possibly the upgrade of existing solutions, in addition to the actual operating system upgrade.
In fact the main driver of IT budget increase during the last 3 years is the need to update outdated infrastructure, and upgrading is part of everyone’s workflow. In the meantime, Ubuntu ESM serves as a vessel to give you the flexibility to schedule infrastructure upgrades while also ensuring the security and continuity of your systems.
Learn more on how other customers leveraged Ubuntu 14.04 ESM in the case studies below.
ESM maintains Interana’s system security while upgrading ›
TIM ensures system security and client confidence with ESM ›