CMU-CS-24-161
Computer Science Department
School of Computer Science, Carnegie Mellon University



CMU-CS-24-161

A Principled Framework for Pliable and
Secure Speculation in Operating Systems

Tae Hoon Kim

M.S. Thesis

December 2024

CMU-CS-24-161.pdf


Keywords: Operating Systems, Security, Speculative Execution, Virtualization

Transient execution attacks present an unprecedented threat to computing systems. Protecting the operating system (OS) is exceptionally challenging because a transient execution gadget in the OS can potentially leak the entire memory.

In this work, we propose Perspective, a principled framework for building pliable and secure speculative execution defenses for the OS. Perspective offers a pliable interface that allows the OS to communicate its security requirements to hardware defenses, enabling tailored protection against transient execution attacks with little performance overhead. The design of Perspective is driven by a taxonomy of transient execution attacks in the OS kernel: (i) active transient execution attacks, where the attacker process exploits its own kernel thread to speculatively execute a transient execution gadget in the kernel, and (ii) passive transient execution attacks, where the attacker coerces the victim process's kernel thread to execute a transient execution gadget. Based on the taxonomy, Perspective introduces Data Speculation Views (DSVs) and Instruction Speculation Views (ISVs), to mitigate active and passive attacks, respectively. DSVs define the ownership of kernel data by a given execution context and block any speculative access to data outside the DSV. ISVs define the set of kernel functions that can be speculatively executed by a given execution context. Any transmitter instructions–whose execution could leak secrets, such as load instructions–that belong to kernel functions outside the ISVs are blocked from speculative execution. ISVs open up new opportunities of (i) swiftly patching gadgets in the OS, (ii) reducing the surface of passive attacks, and (iii) speeding up the process of auditing transient execution gadgets in the OS.

We build Perspective's software components in the Linux kernel and model the hardware components in gem5. We evaluate the security and performance of Perspective on a set of microbenchmarks and datacenter applications. Perspective has an execution overhead over an unprotected kernel of only 3.5% on microbenchmarks and only 1.2% on datacenter applications.

68 pages

Thesis Committee:
Dimitrios Skarlatos (Chair)
Wenting Zheng

Srinivasan Seshan, Head, Computer Science Department
Martial Hebert, Dean, School of Computer Science


Return to: SCS Technical Report Collection
School of Computer Science

This page maintained by reports@cs.cmu.edu