One of the fundamental problems in machine learning and statistics is learn ing generative models of data. Explicit generative models, which model proba-bility densities of data, have been intensively studied in numerous applications.However, it is usually difficult to model complex data, such as natural images, by using combinations of simple parametric distributions. Implicit generative models (IGMs), which model transformations between known source distributions and target distributions to simulate the sampling process without specifying densities explicitly, regain its attention with explosion of interests. With recent success of deep learning, IGMs have yielded impressive empirical performance in different applications.

While there are new algorithms for learning IGMs, its theoretical properties are weakly justified and their relationships with existing methods are under explored. The first thrust of this thesis is to understand statistical guarantees oflearning IGMs. By connecting IGMs with two-sample test, we propose a newgeneric algorithm that can be built on the top of many existing approaches and bring performance improvement over the state-of-the-art. On the other hand, from the perspective of statistical analysis, IGMs, which model transformations, is fundamentally different from traditional explicit models, which makes the existing results not directly applicable. We then study error bounds and sample complexities of learning IGMs taking a step forward in building its rigorous foundations.

In the second part, we shift our focus to different types of data that we are interested in. We develop algorithms for learning IGMs on various data ranging from images, text, to point clouds, by exploiting their underlying structures.Instead of modeling IGM transformations blindly via powerful functions only, such as deep neural networks, we propose to leverage human priors into algorithm designs to reduce model sizes, save computational overhead, and achieve interpretable results. In this thesis, we show an example of incorporating a simple yet fairly representative renderer developed in computer graphics into IGM transformations for generating realistic and highly structured body data, which paves a new path of learning IGMs.

Finally, we study how IGMs can improve existing machine learning algorithms. From its nature of modeling sampling processes, we propose learning powerful kernels via Fourier analysis and IGM sampling. By thinking IGMs as learning transformations, we extend IGMs to broader applications in different domains. In the second example, we present how to learn proximal operators as IGM transformations to solve important linear inverse problems in computer vision. Lastly, we introduce a new way of using IGMs by treating them as auxiliary components to benefit non-generative tasks while the final output of the interest is not the generative models. We present an application of optimizing test power in anomaly detection by constructing a lower bound of test power via auxiliary IGMs.

191 pages

Thesis Committee:
Barnabás Póczos (Chair)
Jeff Schneider
Ruslan Salakhutdinov
Phillip Isola (Massachusetts Institute of Technology)

Roni Rosenfeld, Head, Machine Learning Department
Martial Hebert, Dean, School of Computer Science