|
|
|
|
|
Measuring Fingertip Force by Imaging the Fingernail The goal of this project is to measure fingertip forces by using a multi-camera system to image coloration changes in the fingernail and surrounding skin. Key advantages of this transduction method are that objects do not have to be instrumented with force sensors, and relatively unconstrained grasps can be measured. Grasping studies to date have required the design of special test objects using key-like touch pads instrumented with force sensors, but the new technique now allows the characterization of grasp forces for ordinary objects of arbitrary shape. It also allows the measurement of fingertip forces when regrasping and manipulating objects, where the contact points change in an unpredetermined manner. Key steps are the following.2D to 3D Image Registration [Haptics06] [CVPR07][TBME08]
Elastic Warping [TRO09][WHC07][CVPR07]
We use a Canny edge filter to automatically detect the boundary of the fingernail. However, because of the broken skin around the fingernail, the automatically detected boundary is noisy and can rarely form a smooth curve. We use cubic B-splines to fit the edges and achieve a close-loop contour. The region inside of the contour is the segmented nail.
The nail and the surrounding skin are transformed to the atlas image with boundary-based elastic deformation transformation. We model both the fingernail and surrounding skin regions as elastic sheets that are warped by an external force field applied to the boundaries. Since elastic warping tends to preserve color pattern shapes and the relative position of the patterns, it suits color pattern comparison across subjects.
Weighted Least Squares Model [Haptics06][TBME08]
Generally speaking, regions within the fingernail saturate at lower force levels than regions in the surrounding skin. The front of the fingernail saturates at higher force levels than the middle of the fingernail. The skin at the sides of the fingernail saturates at the highest force levels. The relatively low saturation level of the middle of the fingernail emphasizes the need to image the whole fingernail and surrounding skin. Using the good mesh elements, a generalized least squares estimator is applied to predict fingertip force. Automated Calibration [WHC09][Haptics10] To improve our calibration procedure, we are developing a robot that will apply forces in three directions independently. The z-direction force is supplied by a single motor with a linear stage, on which a platform is mounted. A two-DOF pantograph is attached to the platform, on which is mounted a six-axis force sensor. The fingertip rests against the force sensor, while the first and second links of the finger are held in place by a small finger brace. A series of force controllers determine the input signals to each of the three motors. With this design, the test subject can hold the finger still while the machine provides the desired force, allowing for more methodical collection of calibration data. 
Participants
Collaborators Yu Sun, University of South FloridaMartha Flanders, University of Minnesota John F. Soechting, University of Minnesota This project was supported by NIH Grant 1R21EB004600-01A2. Publications
Videos
|
Fingernail locations will vary depending on the grasp and on the
relative locations of the cameras. As a particular fingernail is
imaged, it will be necessary to correspond points in the image to
a reference image so that calibration results can be applied.
Fingernails are curved, so the analysis of the coloration changes has
to be carried out in 3D. Currently there are many registration
methods relying on naturally present features, such as edges, corners
or texture. However, the fingernail is relative featureless and some
features, related to color, change with force. Pairing of those
features with neighborhood correlation gives a great number of
outliers (wrong pairs). We currently use RANSAC to find the inlier
pairs and use those pairs to calculate the tranform matrix. The 3D
model of the fingernail is obtained with a Bumblebee Stereo camera from
Point Grey Research. 
To compare fingernails from different subjects, finger images are
registered to a finger atlas with simplified reference shapes. The
fingernail is modeled as a disk with 70 pixel radius. The surrounding
skin region is composed of a ring and an isosceles trapezoid.
Ideally, the fingernails are mapped to the fingernail atlas, and the
surrounding skin is mapped to the surrounding skin atlas. To do so,
the fingernails in the reference images need to be segmented from the
surrounding skin.
Our research has identified that certain areas of the fingernail
show a strong linear response of coloration to fingertip force,
others do not. Not just the fingernail areas show this effect,
certain areas of the surrounding skin particularly at the base
of the nail show a strong linear response as well. The location
of the good areas depends on the contact conditions.