Sensor Systems Laboratory

Pre-Touch Sensing with Sea Shell Effect

Summary

Seashell Effect Pre-Touch Sensing is a new form of sensing used to help robots sense the shape and material of objects before they grasp. ''Pretouch'' refers to sensing modalities that are intermediate in range between tactile sensing and vision. The novel pretouch technique presented in this paper is effective on materials that prior pretouch techniques fail on. Seashell effect pretouch is inspired by the phenomenon of ''hearing the sea'' when a seashell is held to the ear, and in particular relies on the observation that the ''sound of the sea'' changes as the distance from the seashell to the head varies. To turn the familiar seashell effect into a sensor for robotic manipulation, a cavity and microphone was built into a robot finger. The sensor detects changes in the spectrum of ambient noise that occur when the finger approaches an object. Environmental noise is amplified most (attenuated least) at the cavity's resonant frequency, which changes as the cavity approaches an object.

More Information

After introducing the sensing modality and characterizing its performance, the paper describes experiments performed with prototype sensors integrated into a Willow Garage PR2 robot's gripper. We explore two primary applications: (1) reactive grasp control and (2) pretouch-assisted grasp planning. For reactive grasp control, we demonstrate the advantage of pretouch over pressure sensing for sensing objects that are too compliant for the PR2's tactile sensors to detect during the grasp execution. We demonstrate the benefit of seashell effect pretouch over prior pretouch methods by demonstrating that the new method succeeds with materials on which prior methods fail. For pretouch-assisted grasp planning, the pointcloud from the PR2's texture-projection / narrow stereo camera is augmented with additional points collected by recording the position of the robot's end effector when the pretouch sensor detects the object. This method can compensate for points that are missing from the pointcloud either because of depth camera failure or occlusion. For this application, pretouch is advantageous over contact sensing because contact sensing tends to unintentionally displace the object.