Person Tracking and Gesture Recognition in Challenging Visibility Conditions Using 3D Thermal Sensing

Ariel Kapusta and Patrick Beeson. Person Tracking and Gesture Recognition in Challenging Visibility Conditions Using 3D Thermal Sensing. In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), New York, New York, August 2016.

Abstract

Many existing person tracking systems are challenged by non-laboratory scenarios, including variable lighting conditions, rain, smoke, tracking distance, and tracking speed. We provide evidence that by using a 3D thermal sensor, a person can be tracked in three dimensions with high success using very simple tracking methods, in many of the challenging lighting conditions and other weather conditions that confound other systems. In support of our claim, we present the PROWL (Perception for Robotic Operation over Widespread Lighting) sensor system, which uses thermal stereo image processing and on-board sensor processing to perform person tracking and gesture recognition. PROWL, using only ICP-based point matching algorithms, obtains 100% person tracking success at 20 frames per second out to 13 meters and zero false-positive/false-negative gesture recognition within 7 meters in all tested scenarios, which includes a sunny outdoor environment, a nighttime outdoor environment, a blackout indoor environment, and a whiteout smoke-filled indoor environment.

Additional Information

Talk slides

Project Videos

BibTeX

@InProceedings{Kapusta-roman-16,
  author =       {Ariel Kapusta and Patrick Beeson},
  title =        {Person Tracking and Gesture Recognition in Challenging
                  Visibility Conditions Using 3D Thermal Sensing},
  booktitle =    {Proceedings of the IEEE International Symposium on Robot and
                  Human Interactive Communication~(RO-MAN)},
  year =         2016,
  address =      {New York, New York},
  month =        {August},
  abstract =     {Many existing person tracking systems are challenged by
                  non-laboratory scenarios, including variable lighting
                  conditions, rain, smoke, tracking distance, and tracking
                  speed. We provide evidence that by using a 3D thermal
                  sensor, a person can be tracked in three dimensions with
                  high success using very simple tracking methods, in many of
                  the challenging lighting conditions and other weather
                  conditions that confound other systems. In support of our
                  claim, we present the PROWL (\textit{Perception for Robotic
                  Operation over Widespread Lighting}) sensor system, which
                  uses thermal stereo image processing and on-board sensor
                  processing to perform person tracking and gesture
                  recognition. PROWL, using only ICP-based point matching
                  algorithms, obtains 100\% person tracking success at 20
                  frames per second out to 13 meters and zero
                  false-positive/false-negative gesture recognition within 7
                  meters in all tested scenarios, which includes a sunny
                  outdoor environment, a nighttime outdoor environment, a
                  blackout indoor environment, and a whiteout smoke-filled
                  indoor environment.},
  bib2html_pubtype ={Refereed Conference},
  bib2html_rescat ={Activity Recognition},
  bib2html_extra_info ={<a href="http://personal.traclabs.com/~pbeeson/talks/Kapusta-roman-16_talk.pdf">
Talk slides</a><br><br><a href="http://personal.traclabs.com/~pbeeson/PROWL/">Project Videos</a>}
}

Download:

[837.3KB pdf]