Decompilation is the process of converting executable programs back into source code, reversing compilation. Decompilation is used in several security-related tasks for which the source code of the program of interest is typically unavailable, including malware analysis, vulnerability research, and patching legacy software. Unfortunately, a significant amount of information is discarded during compilation, which means that information is unavailable for decompilers to use when reconstructing the source code. As a result, decompiled code is missing details like variable names, user-defined types, and comments, and is often structured in substantially different ways. The end result is that decompiled code is substantially more difficult to read than traditional source code, making reverse engineering a painstaking process. The key goal of this thesis is a method to recover these missing details with machine learning while not undermining the stronger correctness properties that deterministic decompilation provides.

In the first thrust, to gain insights into the nature of decompiled code with an eye towards how it can be improved for reverse engineers, we conduct an empirical study of decompiled code. In particular, we build a comprehensive, hierarchical taxonomy of the ways in which decompiled code is unfaithful to the original source code from which it was derived (through compilation and subsequent decompilation). Two key conclusions from an analysis of our taxonomy are (1) a lack of type information, especially user-defined-types, are directly or indirectly involved in over half (8/14) of all top-level issues, and (2) arbitrary textual transformations are required to fix all issues generally.

Leveraging these insights, in the second thrust, we built a neural decompilation tool, IDIOMS, to help address the fidelity issues with decompiled code. Because type information and much of the information needed to rewrite the decompiled code is discarded during compilation, our approach is probabilistic, based on machine learning. IDIOMS consists of a family of fine-tuned language models of code. IDIOMS jointly rewrites decompiled code and provides definitions for all user-defined types used within the rewritten code. IDIOMS takes as input the decompiled form of a function and, because code throughout an executable program can inform the prediction of user-defined types, other nearby functions in the binary.

The final thrust addresses a fundamental limitation of neural decompilation tools like IDIOMS: incorrect predictions (mispredictions). Especially concerning are semantic mispredictions: those that do not faithfully represent the behavior of the underlying binary. Mispredictions are a critical threat to validity for neural decompilers because a misprediction could mislead a reverse engineer into drawing incorrect conclusions about the executable. We introduce a technique, FAULTLESS, that checks for the absence of semantic mispredictions by comparing a neural decompiler's predicted code with the decompiled code, allowing the reverse engineer confidence that the rewriting can (or cannot) be trusted. FAULTLESS can also be applied to evaluate a neural decompiler by comparing the predicted and original code.

113 pages

Thesis Committee:
Claire Le Goues (Chair)
Rohan Padhye
Edward J. Schwartz
Lin Tan (Purdue University)

Nicolas Christin, Head, Software and Societal Systems Department
Martial Hebert, Dean, School of Computer Science

Return to: SCS Technical Report Collection
School of Computer Science

This page maintained by reports@cs.cmu.edu