HYPERBOT – a Benchmarking Testbed for Acquisition of Robot-Centric Hyperspectral Scene and in-Hand Object Data

EasyChair Preprint 8110

Abstract

Robots will benefit from identifying novel objects in their environments through multi-modal sensing capabilities. The overarching goal of this research is to accelerate the multi-modal sensor data collection for general-purpose robots to infer material properties of objects they interact with. To this end, we designed a benchmarking testbed to enable a robot manipulator to perceive spectral and spatial characteristics of scene items. Our design includes the use of a push broom Visible to Near Infrared (VNIR) hyperspectral camera, co-aligned with a depth camera. This system enables the robot to process and segment spectral characteristics of items in a larger spatial scene. For more targeted item manipulation, we integrated a VNIR spectrometer into the fingertips of a gripper. By acquiring spectral signatures both at a distance and at grasp time, the robot can quickly correlate data from the two sensors, each of which contain distinct quantum efficiencies and noise. Our approach to this challenge is a step towards using spectral data for enhanced grasp selection in cluttered environments and automated ground-truthing of hyperspectral sensor data. This paper describes our approach to the design of this benchmarking testbed. The project code and material list are located here: https://github.com/RIVeR-Lab/HyperBot

Keyphrases: Ground-Truth Acquisition, Hyperspectral imaging, Robot Spectroscopy, grasp planning, sensor fusion

@booklet{EasyChair:8110,
  author    = {Nathaniel Hanson and Tarik Kelestemur and Joseph Berman and Dominik Ritzenhoff and Taskin Padir},
  title     = {HYPERBOT – a Benchmarking Testbed for Acquisition of Robot-Centric Hyperspectral Scene and in-Hand Object Data},
  howpublished = {EasyChair Preprint 8110},
  year      = {EasyChair, 2022}}