@device(postscript)
@libraryfile(Mathematics10)
@libraryfile(Accents)
@style(fontfamily=timesroman,fontscale=11)
@pagefooting(immediate, left "@c<Technical Report 1996>",
center "@c<Computer Science>",
right "@c<Carnegie Mellon>")
@heading(Hand Action Perception and Robot Instruction)
@heading(CMU-CS-96-116)
@center(@b(Yunde Jiar, Mark Wheeler, Katsushi Ikeuchi))
@center(March 1996)
@center(FTP: Unavailable)
@blankspace(1)
@begin(text)

It is extrememly popular and much more efficient way for teachers
or instructors at school, work, or play, to teach their students a task
just by demonstrating the task clearly in front of the students. Then
the students replicate the demonstrated task to verify if they are able
to do it successfully.

Based on this observation, we presents a general and robust approach
to hand action perception for automatic robot instruction using depth
image suquences. We aim to build such a robot system which should learn
not only the initial poses and final destinations of the objects and
the orders of assembly, but also should perceive how objects move and
how they are manipulated.

In our system the human instructor must simply demonstrate an assembly
task in front of a vision system in the human world, no dataglove or
special markings are necessary. The recorded image sequences are used
to recover a depth image sequence for model-based human hand and object
tracking to form the perceptual data stream. The data stream is then
segmented and interpreted for generating a task sequence which describes
the human hand action and the relationship between the manipulated object
and the hand. The task sequence might be composed of a series of
subtasks and each subtask involves four phases: approaching, 
pre-manipulating, manipulating and departing.

In this paper, we also discuss a robot system that replicates the
observed task and automatically validates the replication results in
the robot world.

@blankspace(2line)
@begin(transparent,size=10)
@b(Keywords:@ )@c<Hand perception, assembly task, model-based tracking,
depth image sequence, human action interpretation, robot instruction>
@end(transparent)
@blankspace(1line)
@end(text)
@flushright(@b[(60 pages)])