Automatic Hand Features Extraction for Forensic Purposes

This work describes a low computational cost image processing methodology for the identification of hand features for forensic applications – like height, gender, and ethnicity for suspect inference. The proposed methodology provides an adequately tailored sequence of image and geometrical processings on pictures of hands based on critical points like fingertips, finger valleys, and palm centers. As a result, we present the automatic measurements taken on 416 images compared with actual measurements from a manual caliper. The proposed approach leads to a 93.16% correlation (p - value < 0.05) related to the caliper, demonstrating the effectiveness of the technique.

The use of anthropometric features for forensic purposes is a field of research in full development. In several countries, researchers use databases of such features for forensic purposes, for example, measuring hand characteristics to infer height [1-5], gender [6], and even ethnicity [7]. However, similar developments are still absent in Brazil. The first challenge in this regard is the automatic identification of hand features from digital photos.

From the last 30 years we verify that, for most studies, the number of subjects is nearly two hundred, as depicted in Table 1. With the advent of modern Big Data, we can process thousands of data using advanced machine learning techniques to improve correlation near to 100%. So, the proposed work aims to create an approach to simplify data acquisition, providing effective big data solutions in the next stages of this research.

However, these studies achieve superior limits in correlation near 80% and can’t go any further, as presented in Figure 1.

In this sense, the motivation for the present works relies on the belief that by increasing the number of hand features extracted and expanding the number of subjects to a vast amount of people we can dramatically improve the final correlation (above 95%) between hand features and subject height, including the possibility to classify gender and ethnicity.

Based on the first Brazilian hand’s image database with 416 images (89 males and 119 females, both hands) acquired by us in previous work [8], we start the development of a novel approach to fully automate the measurements extraction to avoid a usually amount of labor in the process.

The database

The study was approved by the Ethics Committee of the Federal University of ABC (UFABC) under process number 1,334,138. The sample is made up of students, teachers, and employees from two campuses of the Federal University of São Paulo UNIFESP in the state of São Paulo. Only healthy volunteers aged between 18 and 55 years of both sexes were included in the sample. Volunteers with congenital or acquired malformations of the limbs, head, and spine were excluded, as well as those who underwent hand surgery, and users of lower and upper limb prostheses or spinal implants.

The data was collected from volunteers based on their self-declaration: Sex, Place of Birth, Age (years), Ethnicity and Hand Dominance. Height was measured in centimeters and was defined as the distance from the vertex of the volunteer’s head to the ground. Seven parameters of each hand were measured, including the length of the five fingers (distance between the tip and base of the proximal phalanx of each finger), the length of the palm (distance in a straight line between the distal crease of the wrist joint and the base of the proximal phalanx of the middle finger) and the width of the hand (distance between the radial side of the second metacarpophalangeal joint and the ulnar side of the fifth metacarpophalangeal joint).

Figure 2 shows: the length of the thumbs (CPd and CPe – right and left), indicators length (CId and CIe – right and left), medium fingers length (CMed and CMee – right and left); Ring fingers length (CAd and CAe – right and left); minimum fingers length (CMid and CMie – right and left); palms length (CPMd and CPMe – right and left) and hands width (LMd and LMe – right and left).

A Sanny® portable vertical stadiometer was used to measure the volunteer’s height. To carry out measurements of hand configurations, six reference points were defined. Hand parameters were measured using a digital caliper (Mitutoyo Corp, Kanagawa). Hand images were digitized with an Epson Perfection V370 flatbed scanner (Figure 3).

The sample totaled 427 individuals, 241 (56.4%) being female and 186 (43.6%) being male. The predominant ethnicity was white, which represented 62.3% of the sample and the proportion of individuals destroyed was 91.1%. The average age was 23.8 years for men and 23.7 years for women (p = 0.800). The average height was 175.6 cm for men and 162.4 cm for women (p = 0.001) (Figure 4).

The analysis of variance to compare the hands and sexes of the volunteers showed that males had a greater average of all fingers, length, and width of the palm (p < 0.05) (Figures 5,6).

For the sides of the hands, for the IC variable, only women did not report a statistical difference (p = 0.162). Only in men, the CMe variable showed no difference between the left and right hands (p = 0.999).

Since our database has raw images previously measured manually using a caliper we can validate our results from the automated approach developed.

The proposed approach seeks to attend to two requirements:

• Fast processing: less than a minute to process a pair of hands is the aim since the subjects can wait until the processing is done before they are released;
• Low computational resources demanded: the approach will be executed all over the country by several partners not always ensuring a high-end computation desktop or notebook

The desired measurement set to be extracted by our approach was presented in Figure 7: lengths of little, ring, middle and index fingers, thumbs, and palm length and width, both from the right and left hands.

Image preprocessing

The first stage of the process is the image segmentation (Figure 8b). We use a suitable set of parameters to perform a skin-color segmentation. After that, we compute a binarized image by applying a threshold (Figure 8c). Finally, we perform an edge detection using the Canny Algorithm (Figure 8d).

Palm center finding, finger-tips, and finger-valleys determination

One key point in the hand’s feature extraction is to determine the center of the palm, regardless of the length of fingers or the interference of wrist image. To do this, we use the methodology developed by [9] to compute the morphological medial axis using OpenCV library functions (Figures 9a,9b).

Previously developed by [9], the finger-tips and finger-valleys (Figures 10a,10b) key-points detection use convex-hull primitives from OpenCV.

The middle finger, palm length, other fingers, and palm width determination

From now on we need to follow a particular order to reach all desired measurements since the next one depends on a previous one:

1. Middle finger measurement
2. Palm length measurement;
3. Other finger measurements (little, ring, index, and thumb in any order);
4. Palm width measurement.

Figure 6 presents the middle finger measurement methodology. Starting with the binarized image generated in the earlier preprocessing stage (Figure 11a), we connect the middle fingertip with the center of the palm (Figure 11b).

In the next steps, we create another line connecting the middle fingertip with the finger-valley located between the middle and index fingers. Then we project this new line over the first line (Figure 11c) and the result of this projection is the final measure line (Figure 11d).

Figure 12 introduces the palm length measurement methodology. It is straightforward and consists only of connecting the middle finger basis to the center of the palm and then extending this line by applying a 1.67 factor (a factor determined experimentally using the image database and manual caliper measurements).

Once we have palm length, we also have the wrist key point at the far end of the palm length line. For the next step - measurement of other fingers and thumb - this key point will be crucial.

We start with the palm length line (Figure 13a) and then connect all finger and thumb tips to the wrist key point (all but the middle finger), as shown in Figure 13b. Then we perform similar projections as we did before with the middle finger but choose the nearest finger-valley for each finger and thumb (Figure 13c) and finally achieve the final measures (Figure 13d).

The last measure is the palm width, and its methodology is described in Figure 14. Here we use a more sophisticated methodology than previous sections. It involves:

• Bitwise image mask logical operations;
• Advanced image processing techniques like nearest connected components;
• Classical image processing techniques like erosion and dilatation.

We start with the lines from little and index fingers and then extend its far end by about 30% (Figure 14a). In the next step, we interconnect their fair ends (Figure 14b). Now we need to extend this line on both sides in a factor of about 80% of its original size (40% to each side), as shown in Figure 14c.

For the next step, we need to load the hand edge (presented in the initial preprocessing section, Figure 8d) and mask it to the canvas image (Figure 14d). The result will produce small collision points by the intersection of images.

These small points must be dilated and then submitted to the nearest component analysis to detect their object centers (Figure 14e). Finally, by connecting these calculated centers, we find the palm width measurement.

Statistical processes

We use the bivariate 2-tailed Pearson correlation method. Deviations are shown in Figure 15.