A painting is more than just a picture on a wall; a painting is a process comprised of many intentional brush strokes, leading to a performance far richer than the finaloutput. The shapes of individual strokes are an important component of a painting's style. This is especially true for sparse sketches, where individual strokes are isolated and prominent. Prior work in modeling brush stroke trajectories either does not work with real-world robotics or is not flexible enough to capture the complexity of human-made brush strokes. In this work, we aim to develop a robotic drawing agent with controllable stroke-level style based on human trajectories.

To achieve this, we develop a framework to collect brush trajectories from human artists on a real canvas. We model these trajectories with an autoencoder. Finally, we incorporate the autoencoder into the planning pipeline in the FRIDA robotic painting system [22]. We find that, off-the-shelf, FRIDA's brush stroke renderer struggles or fails to learn the complex trajectories from the human demonstration data, especially with narrow brushes or markers. We present a novel brush stroke renderer that is capable of generalizing to complex, human-made brush strokes while maintaining a small Sim2Real gap.

We conduct a survey on Amazon Mechanical Turk to evaluate our method quantitatively. The results indicate that Spline-FRIDA successfully captures the stroke styles in human drawings. They also suggest that Spline-FRIDA's drawings are more human-like, higher quality, more true to the objective, and more artistic compared to FRIDA's drawings.

41 pages

Thesis Committee:
Jean Oh (Chair)
Jim McCann

Srinivasan Seshan, Head, Computer Science Department
Martial Hebert, Dean, School of Computer Science

Return to: SCS Technical Report Collection
School of Computer Science

This page maintained by reports@cs.cmu.edu