Accelerators and I/O devices perform much of today's computation. Accelerators (e.g., for machine learning, cryptography, and video processing) are now commonplace, and even long-established I/O devices such as network interface cards (NICs) and disks have gained programmability and compute of their own. Yet devices today communicate using specialized interfaces better suited to an era when each device performed a single, fixed function under the CPU's control. Specialized interfaces limit devices in two ways. First, interfaces are specialized to a particular data format, unable to adapt to new capabilities as devices gain them. Second, interfaces are specialized to a particular pair of endpoints: the device and the CPU. As a result, when applications would benefit from devices interacting directly, they must rely on vendor-specific solutions that exist for only a handful of device pairs.

A general-purpose interface could address both limits: it could keep pace as devices gain capabilities, and let any device communicate with any other without the need for vendor-specific solutions. What stands in the way is the long-held assumption that generality costs performance. This dissertation disproves that assumption. Our thesis is that devices can adopt a general-purpose interface without compromising performance–in fact, we find that generality can even improve performance by reducing the CPU's involvement in communication. We pursue generality along two axes: "what" devices communicate and with "whom."

To generalize "what" devices communicate, we introduce Ensō, a streaming abstraction between devices and the CPU. Instead of specializing to the data's format and size, Ensō delivers arbitrary data over a stream. This allows devices to extend their capabilities without shoehorning data into legacy formats–a NIC can use the same interface to deliver raw packets, application-level messages, or bytestreams, depending on the offloads it provides. We show that streaming not only makes the interface format-agnostic but also eliminates the overheads that fixed-size buffers impose, letting applications achieve up to 6x the throughput of a specialized NIC.

To generalize with "whom" devices can communicate, we introduce DCP, a general-purpose peer-to-peer Device Communication Protocol. Although interconnects such as PCIe have long supported peer-to-peer communication, device interfaces are designed for talking to the CPU. This is due to differences in capabilities: the CPU is better at exposing memory and centralizing coordination, and devices are better at accessing memory asynchronously. DCP instead provides a symmetric interface that works the same way regardless of devices' capabilities and allows devices to perform distributed coordination, without the CPU. As a result, DCP cuts latency by 83% compared to CPU-mediated communication, while offering similar performance to specialized interfaces when devices do talk to the CPU.

Together, Ensō and DCP show that devices need not remain workers behind restricted interfaces, but can instead act as peers–enabling a future in which a computer looks less like a centralized hub-and-spoke of CPU and accelerators and more like a distributed system of composable peers.

136 pages

Thesis Committee:
Justin Sherry (Chair)
David G. Andersen
James C. Hoe
Arvind Krishnamurthy (University of Washington)
Aurojit Panda (New York University)

Jignesh Patel, Interim Head, Computer Science Department
Martial Hebert, Dean, School of Computer Science

Return to: SCS Technical Report Collection
School of Computer Science

This page maintained by reports@cs.cmu.edu