Autonomous Observation Tarek M. Sobh UUCS-92-041 December, 1992 Department of Computer Science University of Utah Salt Lake City, UT 84112 USA Abstract We address the problem of observing an agent. We advocate a modeling approach for the visual system and its observer, where a discrete event dynamic system (DEDS) framework is developed and ``events'' are defined as ranges on parameter subsets. The dynamic recursive context for finite state machines (DRFSM) is described with some applications in the inspection and reverse engineering domain. We propose a system for observing a manipulation process, where a robot hand manipulates an object. We recognize the hand/object interaction over time and a stabilizing observer is constructed. Low-level modules are developed for recognizing the events that causes state transitions within the dynamic manipulation system. The work examines closely the possibilities for errors, mistakes and uncertainties in the manipulation system, observer construction process and event identification mechanisms. The DRFSM DEDS systems utilizes different tracking techniques in order to observe and recognize tasks and agents in an {\em active}, {\em adaptive} and {\em goal-directed} manner. _____________________________________________________________________ This work was supported in part by DARPA grant N00014-91-J-4123 and NSF grant CDA 9024721; Air Force AFOSR Grants 88-0244, 88-0296; Army/DAAL Grant 03-89-C-0031PRI; NSF Grants CISE/CDA 88-22719, IRI 89-06770; DARPA Grant N0014-88-0630 and DuPont Corporation. All opinions, findings, conclusions or recommendations expressed in this document are those of the author and do not necessarily reflect the views of the sponsoring agencies.