Computer Science Department
School of Computer Science, Carnegie Mellon University
Building Whole Applications Using Only Programming-by-Demonstration
Richard G. McDaniel
Also appears as CMU-HCII-99-100
This thesis has developed techniques that allow a developer to build complete applications without using a written programming language. The foundation of these techniques is programming-by-demonstration in which a developer shows the system what to do by presenting examples of the desired behavior. The techniques include innovations both in interaction and inferencing.
The interaction techniques developed in this research are designed to give developers the ability to express the application's behavior with appropriate detail. The developer can draw guide objects in the scene that are hidden to the user. These objects express important relationships and hold the application's data. The developer can also use cards and decks to represent collections of data as well as to provide randomness. The developer can give the system hints by pointing out which objects a behavior relies upon. Also, the techniques include an efficient method for demonstrating examples called nudges. Nudges allow the developer to revise behaviors as problems are discovered, and they allow the developer create negative examples as easily as positive examples.
The inferencing techniques allow a broader range of behavior to be generated automatically than prior PBD systems allowed. By using decision tree learning, the system can automatically infer conditional expressions where the objects that are referenced by a behavior do not have to be affected by that behavior. Another technique based on recursive difference methods allows the system to form and revise arbitrarily long chains of expressions.
These techniques are implemented in a tool called Gamut which has been tested in a usability study to gauge the techniques' effectiveness. The test showed that Gamut can be successfully used by nonprogrammers to build complicated behaviors. Gamut provides a rich medium for expressing a developer's intentions with sufficient inferencing power to create interactive software while requiring minimal programming expertise.