Abstract Autonomous systems must be able to survive in adversarial or hostile environ- ments where threats evolve and morph. Under conditions in which a class of ad- versarial agents is novel but rare, these systems must rapidly learn and adapt. We introduce Survival-Critical Machine Learning (SCML), a new ML paradigm that defines how autonomous systems that rely on machine learning can negotiate such adversarial environments. Inspired by the ability of a biological entity's immune system to develop defenses against new viruses, SCML systems leverage the workflow of Live Learning to iteratively improve ML models for threat detection.

Beyond the conceptualization of SCML, the main contributions of this dissertation are an analytical model, a prototype implementation, and experimental results of the SCML design tradeoff space. We evaluate the impact on survivability of the various design parameters and demonstrate the intimate relationship between SCML and Live Learning. Notably, we evaluate the impact of the availability of finite countermeasures (CMs), the CM deployment threshold, the number of deployed systems, and the average threat arrival rate, among others, on the probability of survival of agiven mission duration. We also examine various mission success criteria and the effects of divergent performance metrics between SCML and Live Learning. Additionally, we model SCML as a Markov Decision Process (MDP) to demonstrate how it can be analyzed within existing, well-understood ML frameworks such as MDPs and Reinforcement Learning (RL). We also demonstrate Live Learning's extensibility to domains other than visual data, such as short range radar, critical in many environments where SCML systems will need to operate.

Our experimental results confirm that learning can indeed improve survivability in an SCML system. It further shows that the CM deployment threshold and the number of available CMs have a significant impact on survivability. Allowing flexibility in the CM deployment threshold during the mission enhances such survivability under most conditions. Similarly, Live Learning improves the probability of mission success by increasing the likelihood of accurately classifying actual threats (true positives) and decreasing the likelihood of wasting CMs on non-threats (false positives). By defining an SCML MDP, we also show how an SCML system can optimally adjust its CM deployment threshold as a function of state, defined by the number of remaining CMs and the time until mission completion.

169 pages

Thesis Committee:
Mahadev Satyanarayanan (Chair)
Padmanabhan Pillai
Jeff Schneider
Rashmi Vinayak

Srinivasan Seshan, Head, Computer Science Department
Martial Hebert, Dean, School of Computer Science

Return to: SCS Technical Report Collection
School of Computer Science

This page maintained by reports@cs.cmu.edu