Perception Engine for Activity Recognition and Logging using Manual Procedure Instructions

Patrick Beeson, Nicholas Barrash, and Brian Burns. Perception Engine for Activity Recognition and Logging using Manual Procedure Instructions. In Proceedings of the International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS), Turin, Italy, September 2012.

Abstract

TRACLabs and SRI International are designing a vision recognition system to detect and track crew members, human activities, human-vehicle interaction, and team social interactions for ISS, EVA, and future missions like Deep Space Habitat. We call our proposed system PEARL (Perception Engine for Activity Recognition and Logging). In this paper, we present PEARL using Manual Procedure Instructions. For PEARL-MPI, we integrate a visual recognition system with a procedure execution framework and an ontology that stores the known objects, actions, cameras, and observed object locations in the environment. Using formal procedures constrains the type, location, and duration of possible activities, which improves recognition performance. It also provides a way to demonstrate cooperative human/machine logging of complex, multi-step scenarios in the visual scene.

Additional Information

Selected for oral presentation

BibTeX

@InProceedings{Beeson-isairas-12,
  author =       {Patrick Beeson and Nicholas Barrash and Brian Burns},
  title =        {Perception Engine for Activity Recognition and Logging using
                  Manual Procedure Instructions},
  booktitle =    {Proceedings of the International Symposium on Artificial
                  Intelligence, Robotics and Automation in Space (i-SAIRAS)},
  year =         2012,
  address =      {Turin, Italy},
  month =        {September},
  abstract =     {TRACLabs and SRI International are designing a vision
                  recognition system to detect and track crew members, human
                  activities, human-vehicle interaction, and team social
                  interactions for ISS, EVA, and future missions like Deep
                  Space Habitat.  We call our proposed system
                  PEARL~(Perception Engine for Activity Recognition and
                  Logging).  In this paper, we present PEARL using Manual
                  Procedure Instructions.  For PEARL-MPI, we integrate a
                  visual recognition system with a procedure execution
                  framework and an ontology that stores the known objects,
                  actions, cameras, and observed object locations in the
                  environment.  Using formal procedures constrains the type,
                  location, and duration of possible activities, which
                  improves recognition performance.  It also provides a way to
                  demonstrate cooperative human/machine logging of complex,
                  multi-step scenarios in the visual scene.},
  bib2html_pubtype ={Refereed Conference},
  bib2html_rescat ={Activity Recognition},
  bib2html_extra_info ={Selected for oral presentation}
}

Download:

[469.0KB pdf]