FpVTE2012 – Fingerprint Vendor Technology Evaluation 2012

NISTIR 8034
Fingerprint Vendor Technology Evaluation
Craig Watson
Gregory Fiumara
Elham Tabassi
Su Lan Cheng
Patricia Flanagan
Wayne Salamon
http://dx.doi.org/10.6028/NIST.IR.8034
NISTIR 8034
Fingerprint Vendor Technology Evaluation
Craig Watson
Gregory Fiumara
Elham Tabassi
Su Lan Cheng
Patricia Flanagan
Wayne Salamon
Information Access Division
Information Technology Laboratory
This publication is available free of charge from:
http://dx.doi.org/10.6028/NIST.IR.8034
December 2014
U.S. Department of Commerce
Penny Pritzker, Secretary
National Institute of Standards and Technology
Willie May, Acting Under Secretary of Commerce for Standards and Technology and Acting Director
Fingerprint Vendor Technology Evaluation
Evaluation of Fingerprint Matching Algorithms
NIST Interagency Report 8034
Craig Watson • Gregory Fiumara • Elham Tabassi
Su Lan Cheng • Patricia Flanagan • Wayne Salamon
18 December 2014
iv
FPVTE – F INGERPRINT M ATCHING
Acknowledgements
. Sponsors: The authors would like to thank the sponsors of this project: the Department of Homeland
Security (DHS) and the Federal Bureau of Investigation (FBI).
. Data Providers: The authors would like to thank all the organizations that have shared biometric data
with NIST for this and other evaluations. This data has proven invaluable for benchmarking and advancement of biometric matching technologies through these evaluations. Data used in FpVTE came
from the FBI, DHS, Los Angeles County Sheriff’s Department (LACNTY), Arizona Department of Public
Safety (AZDPS), and Texas Department of Public Safety (TXDPS).
. Participants: The authors would also like to thank the participants for their time and contribution to
this evaluation. We know that preparing the software for the evaluation is not a trivial task and the
validation process can be stressful when problems occur. Thank you for your time and participation as
this evaluation would not be possible without it.
Disclaimer
Certain commercial equipment, instruments, or materials are identified in this paper in order to specify the
experimental procedure adequately. Such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that the materials or
equipment identified are necessarily the best available for the purpose.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
v
Contents
Acknowledgements
iv
Disclaimer
iv
Executive Summary
Caveats
xiii
xv
Release Notes
xvi
1
Introduction
1
2
History and Motivation
3
2.1
NIST Biometric Evaluations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
2.2
Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
3
4
5
6
Data
5
3.1
Classes of Participation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
3.2
Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
3.3
Evaluation Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
3.4
“Size” of the Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
Experiment and Test Protocol
10
4.1
API Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
4.1.1
Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
4.1.2
Finalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
4.1.3
Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
4.2
Test Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
4.3
Biometric Evaluation Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
4.4
Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
4.5
Pre-Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
4.6
Receiving Submissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
Two-Stage Matching
16
5.1
Enrollment Set Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
5.2
Identification — Stage One . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
5.3
Identification — Stage Two . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
Metrics
18
6.1
Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18
6.2
DET Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
6.3
Failure to Extract or Match a Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
6.4
Computational Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
6.5
Ground Truth Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
vi
7
8
9
FPVTE – F INGERPRINT M ATCHING
Accuracy Results
22
7.1
Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
7.1.1
Single-Index Finger Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
7.1.2
Two-Index Finger Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
7.2
Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
7.3
Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
36
Accuracy/Search Time Tradeoff
41
8.1
Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
42
8.2
Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
51
8.3
Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
57
Accuracy Computational Resources Tradeoff
61
9.1
Storage and Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
61
9.1.1
Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
61
9.1.2
Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
66
9.1.3
Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
69
Processing Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
74
9.2
10 Ranked Results
78
11 How Many Fingers are Needed
82
12 FpVTE 2003 Comparison
83
13 Lessons Learned for Large-Scale Testing
86
14 Way Forward
88
14.1 Related Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
88
14.1.1 Forensic Palmprint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
88
14.1.2 Mobile Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
88
14.1.3 Cross-Comparison of Modalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
88
References
89
A Individual Participant FNIR Plots
90
B Combined Class DETs and CMCs
109
C Accuracy Time Tradeoff Detailed Tables with Median Values
114
D Accuracy Time Tradeoff Detailed Tables with Mean Values
128
E Progression for Last Two Submissions
142
F Enrollment Size
153
G Search Template Sizes
160
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
vii
G.1 Mean Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
G.2 Median Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
H Template Creation Times
168
I
Ranked Results
178
J
Relative Combined Results
188
K Relative Accuracy and Number of Fingers
198
K.1 Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
K.2 Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
K.3 Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
K.4 All Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
L Combined Sorted Rankings for a Theoretical Use Case
207
List of Figures
1
NIST Biometric Evaluations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
2
Example of Single-Finger Captures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
3
Example of an Identification Flat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
4
Examples of Live-Scan and Rescanned Ink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
5
NFIQ Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
6
Number of Submissions Received . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
7
Evaluation Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
9
Example of a Flipped Single-Index Finger Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
8
Example of a DET showing a Spike in FPIR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
10
Example of a Flipped Left Slap Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21
11
Example of a Flipped Right Slap Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21
12
Rank-sorted FNIR for Class A — Single Index Finger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
13
DET for Class A — Single Index Finger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24
14
CMC for Class A — Single Index Finger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
15
Rank-sorted FNIR for Class A — Two Index Fingers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
16
DET for Class A — Two Index Fingers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28
17
CMC for Class A — Two Index Fingers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
29
18
Rank-sorted FNIR for Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
19
DET for Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
32
20
CMC for Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
33
21
Rank-sorted FNIR for Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
36
22
DET for Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
23
CMC for Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
38
24
FNIR and Search Time Scatter Plot — Class A — Left Index — Less Than 20-Second Searches . . . . . . . . . . . . . . . .
43
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
viii
FPVTE – F INGERPRINT M ATCHING
25
FNIR and Search Time Scatter Plot — Class A — Left Index — Greater Than or Equal To 20-Second Searches . . . . . . .
43
26
FNIR and Search Time Scatter Plot — Class A — Right Index — Less Than 20-Second Searches . . . . . . . . . . . . . . .
44
27
FNIR and Search Time Scatter Plot — Class A — Right Index — Greater Than or Equal To 20-Second Searches . . . . . .
44
28
FNIR and Search Time Scatter Plot — Class A — Left and Right Index — Less Than 20-Second Searches . . . . . . . . . .
45
29
FNIR and Search Time Scatter Plot — Class A — Left and Right Index — Greater Than or Equal To 20-Second Searches .
45
30
Submission Round Progression — Class A — Left Index — Less Than 20-Second Searches . . . . . . . . . . . . . . . . . .
46
31
Submission Round Progression — Class A — Left Index — Greater Than or Equal To 20-Second Searches . . . . . . . . .
46
32
Submission Round Progression — Class A — Right Index — Less Than 20-Second Searches . . . . . . . . . . . . . . . . .
47
33
Submission Round Progression — Class A — Right Index — Greater Than or Equal To 20-Second Searches . . . . . . . .
47
34
Submission Round Progression — Class A — Left and Right Index — Less Than 20-Second Searches . . . . . . . . . . . .
48
35
Submission Round Progression — Class A — Left and Right Index — Greater Than or Equal To 20-Second Searches . . .
48
36
FNIR and Search Time Scatter Plot — Class B — Left Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
52
37
FNIR and Search Time Scatter Plot — Class B — Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
52
38
FNIR and Search Time Scatter Plot — Class B — Left and Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
53
39
FNIR and Search Time Scatter Plot — Class B — Identification Flats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
53
40
Submission Round Progression — Class B — Left Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
54
41
Submission Round Progression — Class B — Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
54
42
Submission Round Progression — Class B — Left and Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
43
Submission Round Progression — Class B — IDFlat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
44
FNIR and Search Time Scatter Plot — Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
58
45
Submission Round Progression — Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
59
46
FNIR and RAM Used for Enrollment Set Tradeoff Scatter Plot for Class A — Left Index
. . . . . . . . . . . . . . . . . . .
62
47
FNIR and RAM Used for Enrollment Set Tradeoff Scatter Plot for Class A — Right Index . . . . . . . . . . . . . . . . . . .
62
48
FNIR and Search Template Size in RAM Tradeoff Scatter Plot for Class A — Left Index . . . . . . . . . . . . . . . . . . . .
63
49
FNIR and Search Template Size in RAM Tradeoff Scatter Plot for Class A — Right Index . . . . . . . . . . . . . . . . . . .
63
50
FNIR and RAM Used for Enrollment Set Scatter Plot for Class A — Left and Right Index . . . . . . . . . . . . . . . . . . .
64
51
FNIR and Search Template Size in RAM Tradeoff Scatter Plot for Class A — Left and Right Index . . . . . . . . . . . . . .
64
52
Enrollment Size in RAM Compared to On Disk for Class A — Left and Right Index . . . . . . . . . . . . . . . . . . . . . .
65
53
Enrollment Size and Search Template Size in RAM Compared to On Disk for Class A — Left and Right Index . . . . . . .
65
54
FNIR and RAM Used for Enrollment Set for Class B — Identification Flats . . . . . . . . . . . . . . . . . . . . . . . . . . .
67
55
FNIR and Search Template Size in RAM Tradeoff Scatter Plot for Class B — Identification Flats . . . . . . . . . . . . . . .
67
56
Enrollment Size in RAM Compared to On Disk for Class B — Identification Flats . . . . . . . . . . . . . . . . . . . . . . .
68
57
Search Template Size in RAM Compared to On Disk for Class B — Identification Flats . . . . . . . . . . . . . . . . . . . .
68
58
FNIR and RAM Used for Enrollment Set Tradeoff Scatter Plot for Class C — Ten-Finger Plain-to-Plain . . . . . . . . . . .
70
59
FNIR and RAM Used for Enrollment Set Tradeoff Scatter Plot for Class C — Ten-Finger Rolled-to-Rolled . . . . . . . . .
70
60
FNIR and Search Template Size in RAM Tradeoff Scatter Plot for Class C — Ten-Finger Plain-to-Plain . . . . . . . . . . .
71
61
FNIR and Search Template Size in RAM Tradeoff Scatter Plot for Class C — Ten-Finger Rolled-to-Rolled . . . . . . . . . .
71
62
Enrollment Size and Search Template Size in RAM Compared to On Disk for Class C — Plain Impression . . . . . . . . .
72
63
Enrollment Size and Search Template Size in RAM Compared to On Disk for Class C — Plain Impression . . . . . . . . .
72
64
Enrollment Size and Search Template Size in RAM Compared to On Disk for Class C — Rolled Impression . . . . . . . .
73
65
Enrollment Size and Search Template Size in RAM Compared to On Disk for Class C — Rolled Impression . . . . . . . .
73
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
ix
66
FNIR and Template Creation Time Tradeoff Scatter Plot for Class A — Left Index . . . . . . . . . . . . . . . . . . . . . . .
75
67
FNIR and Template Creation Time Tradeoff Scatter Plot for Class A — Right Index . . . . . . . . . . . . . . . . . . . . . .
75
68
FNIR and Template Creation Time Tradeoff Scatter Plot for Class A — Left and Right Index . . . . . . . . . . . . . . . . .
76
69
FNIR and Template Creation Time Tradeoff Scatter Plot for Class B — Identification Flats . . . . . . . . . . . . . . . . . .
76
70
FNIR and Template Creation Time Tradeoff Scatter Plot for Class C — Ten-Finger Plain-to-Plain . . . . . . . . . . . . . .
77
71
FNIR and Template Creation Time Tradeoff Scatter Plot for Class C — Ten-Finger Rolled-to-Rolled . . . . . . . . . . . . .
77
72
Rank-sorted FNIR for All Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
82
73
FpVTE 2003 SST Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
83
74
FpVTE 2003 MST Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
84
75
FpVTE 2003 LST Results
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
84
76
FpVTE 2003 LST Number of Fingers Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
85
77
DET for Participant C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
91
78
DET for Participant D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
92
79
DET for Participant E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
93
80
DET for Participant F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
94
81
DET for Participant G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
95
82
DET for Participant H . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
96
83
DET for Participant I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
97
84
DET for Participant J . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
98
85
DET for Participant K . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
99
86
DET for Participant L . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
87
DET for Participant M . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
88
DET for Participant O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
89
DET for Participant P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
90
DET for Participant Q . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
91
DET for Participant S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
92
DET for Participant T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
93
DET for Participant U . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
94
DET for Participant V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
95
DET for All Participants in All Classes — First Submissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
96
DET for All Participants in All Classes — Second Submissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
97
CMC for All Participants in All Classes — First Submissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
98
CMC for All Participants in All Classes — Second Submissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
99
Relative Combined Results — Left Index — Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
100 Relative Combined Results — Right Index — Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
101 Relative Combined Results — Left and Right Index — Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
102 Relative Combined Results — Left Slap — Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
103 Relative Combined Results — Right Slap — Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
104 Relative Combined Results — Left and Right Slap — Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
105 Relative Combined Results — Identification Flats — Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
106 Relative Combined Results — Ten-Finger Plain-to-Plain — Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
x
FPVTE – F INGERPRINT M ATCHING
107 Relative Combined Results — Ten-Finger Rolled-to-Rolled — Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
108 Relative Accuracy — Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
109 Relative Accuracy — Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
110 Relative Accuracy — Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
111 Relative Accuracy — All Classes (Select Fingers) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
List of Tables
1
List of Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
2
Data Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
3
Evaluation Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
4
Search and Enrollment Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
5
Number of Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
6
Finger Position Swapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
−3
— Class A — Left Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
FNIR @ FPIR = 10
−3
— Class A — Right Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
FNIR @ FPIR = 10
−3
— Class A — Left and Right Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
FNIR @ FPIR = 10
−3
— Class B — Left Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
34
FNIR @ FPIR = 10
−3
— Class B — Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
34
FNIR @ FPIR = 10
−3
— Class B — Left and Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
13
FNIR @ FPIR = 10
−3
— Class B — Identification Flats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
14
FNIR @ FPIR = 10−3 — Class C — Ten-Finger Plain-to-Plain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
39
7
FNIR @ FPIR = 10
8
9
10
11
12
FNIR @ FPIR = 10
−3
— Class C — Ten-Finger Rolled-to-Rolled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
39
16
FNIR @ FPIR = 10
−3
— Class C — Ten-Finger Plain-to-Rolled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
40
17
Median Identification Times for Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
49
18
Median Identification Times for Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
50
19
Median Identification Times for Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
56
20
Median Identification Times for Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
60
21
Ranked Results for Class A — Left and Right Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
79
22
Ranked Results for Class B — Identification Flats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
80
23
Ranked Results for Class C — Ten-Finger Rolled-to-Rolled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
81
24
Median Identification Time Results for Class A — Left Index — Less Than 20-Second Searches . . . . . . . . . . . . . . . 115
25
Median Identification Time Results for Class A — Left Index — Greater Than or Equal To 20-Second Searches . . . . . . 116
26
Median Identification Time Results for Class A — Right Index — Less Than 20-Second Searches . . . . . . . . . . . . . . 117
27
Median Identification Time Results for Class A — Right Index — Greater Than or Equal To 20-Second Searches . . . . . . 118
28
Median Identification Time Results for Class A — Left and Right Index — Less Than 20-Second Searches . . . . . . . . . 119
29
Median Identification Time Results for Class A — Left and Right Index — Greater Than or Equal To 20-Second Searches
30
Median Identification Time Results for Class B — Left Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
31
Median Identification Time Results for Class B — Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
32
Median Identification Time Results for Class B — Left and Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
33
Median Identification Time Results for Class B — Identification Flats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
15
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
120
FPVTE – F INGERPRINT M ATCHING
xi
34
Median Identification Time Results for Class C — Ten-Finger Plain-to-Plain . . . . . . . . . . . . . . . . . . . . . . . . . . 125
35
Median Identification Time Results for Class C — Ten-Finger Rolled-to-Rolled . . . . . . . . . . . . . . . . . . . . . . . . 126
36
Median Identification Time Results for Class C — Ten-Finger Plain-to-Rolled . . . . . . . . . . . . . . . . . . . . . . . . . 127
37
Mean Identification Time Results for Class A — Left Index — Less Than 20-Second Searches . . . . . . . . . . . . . . . . 129
38
Mean Identification Time Results for Class A — Left Index — Greater Than or Equal To 20-Second Searches . . . . . . . . 130
39
Mean Identification Time Results for Class A — Right Index — Less Than 20-Second Searches
40
Mean Identification Time Results for Class A — Right Index — Greater Than or Equal To 20-Second Searches . . . . . . . 132
41
Mean Identification Time Results for Class A — Left and Right Index — Less Than 20-Second Searches . . . . . . . . . . 133
42
Mean Identification Time Results for Class A — Left and Right Index — Greater Than or Equal To 20-Second Searches . . 134
43
Mean Identification Time Results for Class B — Left Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
44
Mean Identification Time Results for Class B — Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
45
Mean Identification Time Results for Class B — Left and Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
46
Mean Identification Time Results for Class B — Identification Flats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
47
Mean Identification Time Results for Class C — Ten-Finger Plain-to-Plain . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
48
Mean Identification Time Results for Class C — Ten-Finger Rolled-to-Rolled . . . . . . . . . . . . . . . . . . . . . . . . . . 140
49
Mean Identification Time Results for Class C — Ten-Finger Plain-to-Rolled . . . . . . . . . . . . . . . . . . . . . . . . . . 141
50
Progression of Identification Timing and Accuracy for Class A — Left Index . . . . . . . . . . . . . . . . . . . . . . . . . . 143
51
Progression of Identification Timing and Accuracy for Class A — Right Index . . . . . . . . . . . . . . . . . . . . . . . . . 144
52
Progression of Identification Timing and Accuracy for Class A — Left and Right Index . . . . . . . . . . . . . . . . . . . . 145
53
Progression of Identification Timing and Accuracy for Class B — Left Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
54
Progression of Identification Timing and Accuracy for Class B — Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . 147
55
Progression of Identification Timing and Accuracy for Class B — Left and Right Slap . . . . . . . . . . . . . . . . . . . . . 148
56
Progression of Identification Timing and Accuracy for Class B — Identification Flats . . . . . . . . . . . . . . . . . . . . . 149
57
Progression of Identification Timing and Accuracy for Class C — Ten-Finger Plain-to-Plain . . . . . . . . . . . . . . . . . 150
58
Progression of Identification Timing and Accuracy for Class C — Ten-Finger Rolled-to-Rolled . . . . . . . . . . . . . . . . 151
59
Progression of Identification Timing and Accuracy for Class C — Ten-Finger Plain-to-Rolled . . . . . . . . . . . . . . . . 152
60
Storage and RAM Requirements for Class A — Left Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
61
Storage and RAM Requirements for Class A — Right Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
62
Storage and RAM Requirements for Class A — Left and Right Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
63
Storage and RAM Requirements for Class B — Identification Flats
64
Storage and RAM Requirements for Class C — Ten-Finger Plain-to-Plain . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
65
Storage and RAM Requirements for Class C — Ten-Finger Rolled-to-Rolled . . . . . . . . . . . . . . . . . . . . . . . . . . 159
66
Mean Search Template Sizes for Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
67
Mean Search Template Sizes for Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
68
Mean Search Template Sizes for Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
69
Median Search Template Sizes for Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
70
Median Search Template Sizes for Class B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
71
Median Search Template Sizes for Class C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
72
Enrollment Time Results for Class A — Left Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
73
Enrollment Time Results for Class A — Right Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
74
Enrollment Time Results for Class A — Left and Right Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
. . . . . . . . . . . . . . . 131
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
xii
FPVTE – F INGERPRINT M ATCHING
75
Enrollment Time Results for Class B — Left Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
76
Enrollment Time Results for Class B — Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
77
Enrollment Time Results for Class B — Left and Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
78
Enrollment Time Results for Class B — Identification Flats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
79
Enrollment Time Results for Class C — Ten-Finger Plain-to-Plain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
80
Enrollment Time Results for Class C — Ten-Finger Rolled-to-Rolled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
81
Ranked Results for Class A — Left Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
82
Ranked Results for Class A — Right Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
83
Ranked Results for Class A — Left and Right Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
84
Ranked Results for Class B — Left Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
85
Ranked Results for Class B — Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
86
Ranked Results for Class B — Left and Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
87
Ranked Results for Class B — Identification Flats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
88
Ranked Results for Class C — Ten-Finger Plain-to-Plain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
89
Ranked Results for Class C — Ten-Finger Rolled-to-Rolled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
90
Operational Ranking for Class A — Left Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
91
Operational Ranking for Class A — Right Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
92
Operational Ranking for Class A — Left and Right Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
93
Operational Ranking for Class B — Left Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
94
Operational Ranking for Class B — Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
95
Operational Ranking for Class B — Left and Right Slap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
96
Operational Ranking for Class B — Identification Flats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
97
Operational Ranking for Class C — Ten-Finger Plain-to-Plain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
98
Operational Ranking for Class C — Ten-Finger Rolled-to-Rolled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
xiii
Executive Summary
FpVTE was conducted primarily to assess the current capabilities of fingerprint matching algorithms using operational
datasets containing several million subjects. There were three classes of participation that examined one-to-many identification using various finger combinations from single finger up to ten fingers. Class A used single-index finger capture
data and evaluated single index finger (right or left) and two index finger (right and left) identification. Class B used
identification flat (IDFlat) captures (4-4-2; left slap, right slap, and two thumbs simultaneously) and evaluated ten-finger,
eight-finger (right and left slap), and four-finger (right or left slap) identification. Class C used rolled and plain impression
(4-4-1-1; left slap, right slap, left thumb, and right thumb) captures and evaluated ten-finger rolled-to-rolled, ten-finger
plain-to-plain, and ten-finger plain-to-rolled identification. Enrollment sets used for one-to-many identification varied in
size from 5 000 up to 5 000 000 enrolled subjects. Any segmentation of four-finger slap images or two-thumb captures was
performed by the submitted software. All data used was sequestered operational data that was not shared with any of the
participants.
The evaluation allowed each participant to make two submissions per class (A, B, and C) of participation over three
rounds. After each of the first two rounds of submissions, feedback was provided to the participants and they were
allowed to evaluate their performance, make adjustments to their submissions, and resubmit for the next round. The
results of the third and final round of submissions are reported in this document.
The evaluation was conducted at the National Institute of Standards and Technology (NIST) using commodity NISTowned hardware. Participant submissions were compliant to the testing Application Programming Interface (API), which
were linked to a NIST-developed test driver and run by NIST employees. All submissions went through validation testing
to ensure that results generated on NIST’s hardware matched results participants generated on their own hardware.
This was the first large-scale one-to-many fingerprint evaluation since FpVTE 2003. In 2003, participants brought their
own hardware to NIST to process the evaluation data. The datasets in 2003 had approximately 25 000 subjects and required millions of single subject-to-subject matches. The current FpVTE used a testing model closer to real one-to-many
identification systems by allowing the submitted software to control how it does the one-to-many search and return a
candidate list of potential matches. The number of subjects used was also significantly higher, as the current FpVTE had
≈ 10 million subjects in the testing datasets.
The results in this report are based on 30 000 (10 000 mates and 20 000 nonmates) search subjects. There will be an additional report with results (lower errors rates) using 350 000 (50 000 mates and 300 000 nonmates) search subjects.
In addition to measuring current performance capabilities of one-to-many identification algorithms, FpVTE was conducted
to:
. study open-set identification versus enrolled sample sizes extending into the multiple millions;
. provide a testing framework and API for enrollment sizes that must be spread across the memory of multiple compute nodes;
. evaluate on operational datasets containing newer data from live-scan ten-finger “identification flat” capture systems, other live-scan capture devices (e.g., single-finger and multi-finger), and historically significant scanned inked
fingerprints;
. analyze one-to-many identification accuracy, speed, template size, number of fingers, enrollment set sizes, and computational resources;
. create a fingerprint testing data repository with a vast majority of data errors corrected.
FpVTE was not intended to:
. measure performance of an operational Automated Fingerprint Identification System (AFIS);
. evaluate scanners or acquisition devices;
. evaluate latent fingerprint or mobile-captured data.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
xiv
FPVTE – F INGERPRINT M ATCHING
Key results from FpVTE were:
. Fingerprint Identification Accuracy: The most accurate fingerprint identification submissions achieved false negative identification rates (FNIR, or “miss rates”) of 1.9% for single index fingers, 0.27% for two index fingers, 0.45%
for four-finger identification flats (IDFlats), 0.15% for eight-finger IDFlats, 0.09% for ten-finger IDFlats, 0.1% for tenfinger rolled-to-rolled, 0.13% for ten-finger plain-to-plain, and 0.11% for ten-finger plain-to-rolled. These numbers
are reported at a false positive identification rate (FPIR) of 10−3 . 30 000 search subjects were used for these results
(10 000 mates and 20 000 nonmates). The number of enrolled subjects used for single index fingers was 100 000, 1.6
million for two index fingers, 3 million for IDFlats, and 5 million for ten finger plains and rolled. A larger search set
(50 000 mates and 300 000 nonmates) is being completed so that an even smaller FPIR can be reported. Those results,
when available, will be included in an additional report. Section 7.
. Accuracy versus Speed: The fastest submissions were not the most accurate. The most accurate submissions showed
the ability to decrease search times with minimal loss in accuracy. The format of this evaluation may not have fully
explored the lower limit of search speed with minimal loss in accuracy. Section 8 and Appendix E.
. Number of Fingers: Not surprisingly, using more fingers improved accuracy. In fact, the most accurate results were
achieved with ten fingers, searching against the largest enrollment sets of 3 and 5 million subjects. An interesting
result that needs further study and analysis is that two-index finger accuracy was better than four-finger IDFlats.
Based on some analysis of missed mates, it appears image quality may have played some roll in this result. Section 11.
. Computation Resources: The most accurate submissions were able to achieve their results with similar Random
Access Memory (RAM) usage to other submissions. These same accurate submissions typically took longer to enroll
(i.e., extract features) the fingerprint images used in the evaluation. Section 9.
. Candidate Lists: For most top performers, a majority of the time, the mate appeared within the top three candidates
of the candidate list or did not appear at all. Section 7.
. Ranked Results: Results that group FNIR, enrollment time, search time, and RAM usage all in a single table are
shown in Section 10 and Appendix I. The tables are rank-sorted on FNIR, but include ranks in all the other categories
for cross-category comparison.
. Number of Subjects in Enrollment Set: Results for different enrollment set population sizes will not be available
until the larger search sets are complete, but the initial results across classes of participation showed that eight- and
ten-finger accuracy was superior versus larger enrollment sets. In fact, the most accurate identification results were
always achieved with the ten-finger search sets. Section 7.
. Data Type: There were three classes of participation in which data from single fingers, IDFlats, and “legacy” tenfinger rolled and plain impression data types were evaluated. Results of IDFlats and ten-finger rolled and plain
showed little variation in the accuracy of the top-performing submissions. This means the best performers could
tune their submissions to accurately match all the data types evaluated. Section 10.
. Accuracy Gap: The “gap” between the most accurate submissions and the “next tier” appears to be much closer
than in FpVTE 2003. Sections 10 and 12.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
xv
Caveats
1. Specific nature of the biometric data: The absolute error rates quoted here were measured over a very large fixed
corpus of operational fingerprint images. The error rates measured here are realistic if the submissions are applied
to this kind of data. However, in other applications, the applicability of the results may differ due to a number of
factors legitimately not reflected in the FpVTE experimental design. Among these are:
. how much slap fingerprint segmentation errors contribute to core matching accuracy;
. algorithmic limitations caused by the FpVTE API;
. unknown bugs in the submission;
. image quality from using a different data source.
2. Not an Automated Fingerprint Identification System (AFIS) test: While this evaluation is intended to measure the
core capabilities of matching algorithms in a large one-to-many scenario, it is not intended to be a full assessment of
an operational AFIS.
3. Timing of the submissions: While every attempt was made to perform timing on the exact same hardware with the
exact same conditions, it is possible that certain functions of the operating system could have had an unintended
negative effect on timing results. The timing operations reported for each submission submitted were performed in
the exact same manner. Generous “cutoff” times were employed to prevent wasting compute cycles on submissions
whose API functions never returned a value.
4. Aggregate finger positions: When reporting estimated template RAM usage statistics, participants were permitted
to return an aggregate template size when the input image contained more than one finger (e.g., “slaps”). Any
statistics over these types of images were reported as-is and the values were not divided by the number of fingers
expected for the capture type. While these participants are not denoted in the report, the actual RAM usage may be
a better statistic for comparison.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
xvi
FPVTE – F INGERPRINT M ATCHING
Release Notes
All FpVTE related reports, drafts, announcements, and news items may be found on the website
http://fingerprint.nist.gov.
. Application Program Interface and Test Plan: The FpVTE API [15] contains additional details about creating a
submission compatible with the FpVTE test driver. All submissions tested in FpVTE from 2012-2014 were fully
compatible with the FpVTE API and linked without modification to the FpVTE test driver.
. Appendices: Appendix A has full-scale plots for individual participant results that some may prefer to the grouped
plots in the main body of the report. Appendices B through L have complete sets of tables for various analyses to
help reduce the number of tables in the main body of the report.
. Submission identifiers: Throughout this report, submissions are identified by letter code. For reference, the letters
are associated with the providers’ names in a running footnote.
. Typesetting and Graphics: Virtually all of the content in this report was produced automatically. This involved the
R graphs directly from the FpVTE test driver’s output. Other
use of scripting tools to generate LATEX content and graphics were produced with TikZ. Use of these technologies improved timeliness, flexibility, maintainability, and
reduced transcription errors.
. Evaluation Data Ground Truth: Unknown mates within and across datasets create a significant problem and time
delay for large one-to-many data testing.
. Contact: Correspondence regarding this report should be directed to FPVTE at NIST dot GOV.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
1
1
Introduction
In 2012, NIST launched a new Fingerprint Vendor Technology Evaluation (FpVTE) with two main goals. The first goal
was to assess the current capabilities of matching algorithms using operational datasets with several million subjects. The
second goal was to evaluate different operational considerations that could impact matching accuracy. These considerations included number of fingers used, data types (live-scan, single-finger capture, slap capture requiring segmentation,
and rolled), number of enrolled subjects, and matching speeds.
Evaluating biometric capabilities is an important task, particularly for fingerprint identification, given its widespread
applications. Large-scale evaluations of core accuracy and functionality of biometric recognition algorithms using operational data will not only reveal the capabilities of the current state of the art but can also identify the limitations and
gaps of the current algorithms. The former sets realistic operational expectations and the latter directs future research to
improve and enhance current technologies.
FpVTE was conducted by NIST and sponsored by the Department of Homeland Security (DHS) and the Federal Bureau
of Investigation (FBI). All work was performed at the NIST Gaithersburg facility using hardware owned by NIST. FpVTE
includes three rounds of submissions, with participants making algorithm adjustments based on performance reports after
the first two submissions. Participant submissions were required to conform to the test plan API [15].
The evaluation had three classes of participation. Class A used single-index finger capture data and evaluated single
index fingers searched against 5 000 up to 100 000 enrolled subjects (“single-finger identification”), and two index fingers
searched against 10 000 up to 1 600 000 enrolled subjects (“two-finger identification”). Class B used IDFlat captures (44-2) and evaluated ten-finger, eight-finger (right and left slap), and four-finger (right or left slap) identification searched
against 500 000 up to 3 000 000 enrolled subjects. Class C used rolled and plain impression (4-4-1-1) captures and evaluated
ten-finger rolled impression, ten-finger plain impression, and ten-finger plain impression matched to rolled impression
identification searched against 500 000 up to 5 000 000 enrolled subjects. Any segmentation of four-finger slap images or
two-thumb captures was performed by the submission.
Biometric identification is defined as, “[the] process of searching against a biometric enrollment database to find and return
the biometric reference identifier(s) attributable to a single individual” [7]. Biometric identification is broadly categorized
into closed-set and open-set identification.
Closed-set identification refers to cases where all searches have a corresponding enrolled mate in the biometric enrollment
database. An example of a closed-set identification application is a cruise ship on which all passengers are enrolled. The
outcome of a closed-set identification subsystem is a candidate list that contains the identity of one or more enrolled
individuals whose enrolled samples are most similar to the search (query) sample. Ideally, the correct mate appears in
the first rank. As such, the primary accuracy metric for closed-set identification is hit rate (or its complement, miss rate =
1.0 − hit rate), which is the fraction of times the system returns the correct identity within the specified top ranks.
In open-set identification, not all searches have a corresponding enrolled mate in the biometric enrollment database [3].
The expected outcome of an open-set identification subsystem is a candidate list of L closest (or most similar) enrolled
identities when the search sample is from an enrolled individual, or an indication that the search sample is from an
individual not in the biometric enrollment database. Therefore, primary accuracy metrics for an open-set identification
are false positive identification (false alarm or Type I error) rate and false negative identification (miss or Type II error)
rate. These metrics are described in Section 6.
Closed-set identification applications are very limited because in the majority of real-world identification applications,
not all individuals are or can be enrolled. Most real-world biometric identification applications, such as searches against a
watch-list or searches for first-time arrestees, are open-set identification. For that reason, FpVTE only evaluated open-set
identification algorithms.
This document reviews metrics for evaluating the performance of open-set identification algorithms and reports performance for submissions from 18 participants. Three participated in Class A only and the other 15 submitted for all three
classes. Four participants withdrew before making a submission for testing. Table 1 shows the list of participants and in
which classes they made submissions for evaluation. These participants are referenced at the bottom of every page with
their assigned identification letter.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
2
FPVTE – F INGERPRINT M ATCHING
ID
Name
Participation Class
C
D
E
F
G
H
I
J
K
L
M
O
P
Q
S
T
U
V
afis team
3M Cogent
Neurotechnology
Papillon
Dermalog
Hisign Bio-Info Institute
NEC
Sonda
Tiger IT
Innovatrics
SPEX
ID Solutions
id3
Morpho
Decatur Industries
BIO-key
Aware
AA Technology
A, B, C
A, B, C
A, B, C
A, B, C
A, B, C
A, B, C
A, B, C
A, B, C
A
A, B, C
A, B, C
A, B, C
A
A, B, C
A, B, C
A
A, B, C
A, B, C
Table 1: Participant IDs, names, and classes of submission for evaluation.
Section 2 has some history on fingerprint evaluations performed at NIST. Section 3 describes the data used in FpVTE.
Section 4 describes the protocol used for submission acceptance and testing. Section 5 gives details on the two-stage
matching approach used in FpVTE. Sections 6 through 11 talk about the metrics used to measure accuracy and report the
results from testing. While recognition error rates are important and widely reported, computational resources required
by algorithms are a significant aspect of performance, especially for large-scale operations. To that end, we report the
computation time, storage requirements, and their accuracy tradeoffs for each of the submissions. Finally, Section 12
examines some FpVTE 2003 results, then Section 13 and Section 14 talk about some lessons learned and future plans.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
2
2.1
3
History and Motivation
NIST Biometric Evaluations
The first one-to-many fingerprint evaluation conducted at NIST was FpVTE 2003. This evaluation required that participants bring their own hardware and software to NIST for the evaluation. NIST supplied the data and retained all
matching results for final analysis. FpVTE 2003 had three classes of participation: Small-Scale, Medium-Scale, and LargeScale. Small-Scale testing used 1 000 single-finger capture images resulting in 1 million subject-to-subject comparisons.
Medium-Scale testing used 10 000 single-finger captures resulting in 100 million comparisons. Large-Scale testing used
25 000 subjects and various combinations of fingers resulting in 1.044 billion comparisons. The evaluation had 18 participants that submitted 34 systems for testing.
NIST has conducted several fingerprint-related evaluations in the last decade (Figure 1). The first fingerprint evaluation
was called Proprietary Fingerprint Template 2003 (PFT 2003). PFT 2003 was a one-to-one matching evaluation that looked
at the core matching capabilities of fingerprint matching software. It did not evaluate one-to-many capabilities. In 2010,
NIST changed the name of PFT 2003 to PFTII, utilizing newer, larger datasets and reporting information on timing and
template sizes, in addition to accuracy.
Minutiae Exchange (MINEX), also a one-to-one matching evaluation, began in 2005. MINEX was started to support testing
of fingerprint matching technologies using INCITS 378 standard interoperable templates [6]. About a year later, Ongoing
MINEX was created to support Personal Identity Verification (PIV) by establishing guidelines and measuring accuracy for
interoperable template encoders and matchers.
In addition to fingerprint-related evaluations, NIST has performed evaluations for other biometrics such as face and iris.
Additional information and links for all the NIST biometric related evaluations can be found on the NIST biometric evaluations website [5].
FRVT
SlapSeg
FRVT
Biometric
Usability
ICE
2002
2004
2006
IREX I
IQCE
MBE
PFTII
FpVTE
FRVT
IREX IV
FIVE
Tatt-C
2008
2010
2012
2014
2003
2005
2007
2009
2011
FpVTE
PFT
MINEX
FRGC
ICE
ELFT
MINEX II
MBGC
SlapSeg II
ELFT
MBE
IREX III
NFIQ2
2013
Figure 1: NIST biometric evaluations.
2.2
Purpose
One of the main purposes of this FpVTE evaluation was to provide a refresh on the testing performed in 2003 and allow
an opportunity for participation by organizations that missed the previous evaluation. There had been many inquiries in
the past several years on when NIST would perform a similar evaluation. Additionally, the dataset size for FpVTE 2003
was around 25 000 total subjects. The current FpVTE testing used enrollment sets ranging from 10 000 subjects to 5 million
subjects.
NIST has already conducted one-to-many biometric evaluations with enrollment set sizes over 1 million subjects for other
modalities (e.g., Face Recognition Vendor Test (FRVT)/Multiple Biometric Evaluation (MBE) for face and Iris Exchange
Evaluation (IREX III) for iris), in which the sizes of the enrollment templates allowed the entire enrollment set to fit into
the RAM of a single compute node. FpVTE was the first biometric evaluation at NIST that added the ability to partition
the enrollment set across multiple compute nodes, expanding the possibilities in size and breadth of enrollment sets. In
addition to broadening the enrollment set, FpVTE strived to:
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
4
FPVTE – F INGERPRINT M ATCHING
. assess the current performance of one-to-many fingerprint matching software using operational fingerprint data;
. provide a testing framework and API for enrollment sizes that must be spread across the memory of multiple compute nodes;
. support US Government and other sponsors in setting operational thresholds;
. evaluate on operational datasets containing newer data from live-scan ten-finger IDFlat capture systems, other livescan capture devices (e.g., single-finger and multi-finger), and historically significant scanned inked fingerprints;
. analyze one-to-many identification accuracy versus, speed, template size, number of fingers, enrollment set sizes,
and computational resources.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
3
3.1
5
Data
Classes of Participation
FpVTE was separated into three classes of participation: A, B, and C. All participants were required to make a Class A
submission. Along with the Class A submission, a participant could additionally participate in Class B, or both Class B
and Class C. These were the only three participation combinations available.
Images were captured via live-scan sensor and rescanned ink. A live-scan sensor refers to the type of sensor that digitally
records the friction ridges of a finger through techniques such as electrical or optical sensing. Scanned ink is the process of
creating a digital image by using an image scanner to optically capture from paper images of friction ridges created by a
finger covered with ink.
Class A consisted of live-scan single-finger captures of the left and right index fingers (Figure 2).
Figure 2: Example of single-finger captures of left and right index fingers.
Class B consisted of live-scan IDFlats, which captured left and right four-finger slaps and simultaneous left and right
thumbs, known as “4-4-2” (Figure 3).
Figure 3: Example of an identification flat (4-4-2) capture of a left and right slap and left and right thumbs.
Class C consisted of the rolled and plain impressions from a more traditional 14 print card/record and was a mix of
live-scan and rescanned ink (Figure 4).
Figure 4: Example of live-scan and rescanned ink 14 print card/record that include rolled and plain impressions.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
6
3.2
FPVTE – F INGERPRINT M ATCHING
Datasets
The evaluation datasets used in FpVTE were from anonymized operational datasets and made available to NIST for fingerprint evaluations. The datasets are for government use only and will not be released to the public. The datasets will,
to the extent permitted by law, be protected under the Freedom of Information Act (5 U.S.C. § 522) and the Privacy Act (5
U.S.C. § 522a) as applicable.
The datasets were comprised of several fingerprint impression types, including rolled, multi-finger plains, and singlefinger plains. Rolled images were all individual captures that attempted to record the full width of the fingerprint by
rolling from side-to-side during capture. Multi-finger plains captured the four right and four left fingers at the same time.
For identification flats, the two thumbs were captured at the same time. Single-finger plains were individual captures of
the right and left index fingers on a single-finger capture device.
Many of the datasets were larger samples of data used in previous NIST evaluations, such as PFT, MINEX, NIST Fingerprint Image Quality (NFIQ), and FpVTE 2003. The single-finger capture and identification flat fingerprint images were
provided by DHS. The ten-finger rolled and slap fingerprint images included data from the FBI, DHS, Los Angeles County
Sheriff’s Department (LACNTY), Arizona Department of Public Safety (AZDPS), and Texas Department of Public Safety
(TXDPS). Table 2 shows the source of data for each class in the evaluation.
Class
Dataset
Enroll Mate
Enroll Nonmate
Search Mate
Search Nonmate
A
VISIT-I/POEBVA
DHS2
VISIT-II
25%
25%
50%
5%
5%
90%
25%
25%
50%
25%
25%
50%
B
VISIT-II
PDR_IDF
96%
4%
85%
15%
96%
4%
62.5%
37.5%
C
AZDPS
INSBEN
LACNTY
TXDPS
PDR-Roll
33.3%
0%
33.3%
0%
33.3%
4%
11.5%
30%
11.5%
43%
33.3%
0%
33.3%
0%
33.3%
18.75%
12.5%
18.75%
12.5%
37.5%
Table 2: Percentage of data used from each source.
All images in the datasets were 8-bit grayscale. Images were previously compressed using Wavelet Scalar Quantization
(WSQ) compression [2], but were passed to the submitted software as reconstructed raw pixel images, decompressed using
libwsq from NIST’s NIST Biometric Image Software (NBIS) distribution [14]. All images were scanned at 500 pixels per
inch. The dimensions of the images varied, but were provided as input information to the participant’s submission. The
distribution of NIST Fingerprint Image Quality (NFIQ) algorithm [12, 13] values for the evaluation datasets is shown in
Figure 5. NIST Fingerprint Segmentation algorithm (NFSEG) was used to segment slap impression images into individual
fingers before computing NFIQ values.
Multiple-finger plain captures were not segmented. Submissions were required to perform segmentation of fingerprints,
if necessary. Subjects with missing fingers were not removed from the dataset.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Class
Fingers
Type
1
2
3
4
5
A
L/R Index
L/R Index
Enroll
Search
42.8%
39.5%
32.5%
32.9%
20.2%
20.7%
2.2%
2.2%
2.3%
4.7%
B
Left Slap
Right Slap
L/R Slap
IDFlats
Search
Search
Search
Search
44.9%
50.2%
47.6%
48.3%
26.8%
24.4%
25.6%
25.7%
16.0%
13.0%
14.5%
14.2%
6.9%
5.8%
6.3%
6.4%
5.4%
6.6%
6.0%
5.4%
Ten Plain
Ten Rolled
Ten Rolled
Search
Enroll
Search
47.8%
34.2%
35.4%
26.6%
19.5%
21.3%
16.2%
25.0%
23.0%
5.5%
8.6%
9.1%
3.9%
12.7%
11.2%
C
7
100%
75%
50%
25%
0%
L/R Index Left Slap Right Slap L/R Slap
ID Flats
Ten Plain Ten Rolled
Finger
NFIQ
1
2
3
4
5
Figure 5: NFIQ distribution for datasets, after segmentation, where 1 is highest quality and 5 is lowest quality.
3.3
Evaluation Scenarios
The three classes of participation had various data type and fingerprint combinations
that could be evaluated as summarized in autoreftab:data-scenarios. The contents of
Table 4 shows details of the search and enrollment sets used during the evaluation for
each class of participation. The final column of Table 4 shows the various sizes of the
datasets that were tested. The subjects reserved for the search sets contained 200 000
with a known mate and 400 000 with no mate in the enrollment set. This report is based
on a random sample of 10 000 mated and 20 000 nonmated searches from the full search
set.
Class
S1 x E1
A
S2 x E2
S3 x E3
S4 x E4
B
Table 3 shows the various combinations in which the search sets were searched against
the enrollment sets listed in Table 4. Those combinations are also described for each
class as follows.
S5 x E4
S6 x E4
S7 x E4
S8 x E5
C
. Class A — Index Fingers
Scenario
S9 x E5
S9 x E6
– One plain index finger searched against an enrollment set of one plain index
Table 3: Evaluation scenarios.
fingers. The plain images were from single-finger captures of left and right
Refer to Table 4 for descriptions
index fingers.
of the search and enrollment set
– Two plain index fingers searched against an enrollment set of two plain index codes.
fingers. The plain images were from single-finger captures of left and right index fingers.
. Class B — Identification Flats
– Four-, eight-, and ten-finger identification flats (4-4-2, left/right four-finger plain impressions, and a two-thumb
plain impression) searched against an enrollment set of ten-finger identification flats. Any segmentation was
performed by the submission.
. Class C — Ten-Finger Rolled/Slap
– Ten-finger rolled impressions searched against an enrollment set of ten-finger rolled impressions.
– Ten-finger plain impressions searched against an enrollment set of ten-finger plain impressions. Plain impression images were 4-4-1-1 (left/right four-finger plain impression and left/right single-thumb plain impressions). Any segmentation was performed by the submission.
– Ten-finger plain impressions searched against an enrollment set of ten-finger rolled impressions. Plain impression images were 4-4-1-1 (left/right four-finger plain impressions and left/right single-thumb plain impressions). Any segmentation was performed by the submission.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
8
FPVTE – F INGERPRINT M ATCHING
Class
Search
A
Set
Description
# Images
# Fingers
Impression
S1
Right Index
1
1
Plain
S2
Left Index
1
1
Plain
S3
Left and Right Index
2
2
Plain
S4
Right Slap IDFlat
1
4
Plain
S5
Left Slap IDFlat
1
4
Plain
S6
Left and Right Slap IDFlat
2
S7
Identification Flats
3
S8
Ten Finger Rolled
10
S9
Ten Finger Plain
4
E1
Right Index
1
1
Plain
E2
Left Index
1
1
Plain
E3
Left and Right Index
2
2
Plain
E4
Identification Flats
3
E5
Ten Finger Rolled
10
E6
Ten Finger Plain
4
B
C
Enrollment
A
B
8
(4, 4)
10
(4, 4, 2)
10
10
(4, 4, 1, 1)
10
(4, 4, 2)
10
Plain
Plain
Rolled
Plain
Plain
Rolled
C
10
(4, 4, 1, 1)
Plain
# Subjects
Mate: 10 000
Nonmate: 20 000
Mate: 10 000
Nonmate: 20 000
Mate: 10 000
Nonmate: 20 000
Mate: 10 000
Nonmate: 20 000
Mate: 10 000
Nonmate: 20 000
Mate: 10 000
Nonmate: 20 000
Mate: 10 000
Nonmate: 20 000
Mate: 10 000
Nonmate: 20 000
Mate: 10 000
Nonmate: 20 000
10 000
100 000
10 000
100 000
100 000
500 000
1 600 000
500 000
1 600 000
3 600 000
500 000
1 600 000
3 000 000
5 000 000
500 000
1 600 000
3 000 000
5 000 000
Table 4: Search and enrollment datasets used in FpVTE. Class refers to the class of participation, as defined in Subsection 3.1. Set is an identifier used to uniquely identify the various FpVTE datasets in a concise manner. Description indicates
the finger combinations composing each dataset. # Images specifies the number of images per subject. # Fingers shows
the maximum number of fingers per subject. If a subject’s fingers were spread across multiple images, the maximum
number of fingers for each image is also shown. Impression refers to the impression type of the imagery in each dataset.
Impressions could be either plain or rolled, as defined in Subsection 3.2. # Subjects shows the number of subjects in each
dataset. For ’Search’ datasets, the value has been split into mate and nonmate subjects, referring to whether or not a known
mate for the subject exists in the corresponding enrollment set. For ’Enrollment’ datasets, the # Subjects show the various
enrollment set sizes planned for evaluation.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
3.4
9
“Size” of the Test
The overall size of the test had implications on total run time. Table 5
shows the number of enrollments and comparisons that were completed
for each submission. Time limits allowed for a maximum of 3 seconds
per finger for each enrollment, and up to 500 seconds for Class A and 90
seconds for Class B and C to complete each search. Participants could
make two submissions for each class of participation. Three of the 18
participants submitted for Class A only, while the other 15 submitted
for all three classes, creating a total of 96 submissions analyzed in this
report. There were three rounds of submissions, with participants receiving results after the first two submissions to analyze before making
their next submission. Most submissions shared enrollment templates
between both submissions within a class, but two or three submissions in
each round required generating enrollment templates separately.
Class
Enrollments
Performed
Subject to Subject
Comparisons
A
5.8
54
B
45.5
362
C
113
452
Table 5: Approximate number (in millions) of
enrollments and comparisons that were completed to evaluate each submission for that
class of participation.
One unexpected event was that the majority of the submissions required re-enrolling the data for each round of submissions. Each round of submissions required approximately 2.5 billion fingerprint enrollments. It took two to three months
for each round of submissions to enroll all the data for all the submissions before searching could begin. The 30 000 (20 000
nonmate and 10 000 mate) searches took another one to two months to complete for each round of submissions for all
classes across all submissions and finger combinations. The first two rounds resulted in approximately 43 trillion subjectto-subject comparisons, and the final round will have approximately 470 trillion subject-to-subject comparisons. The final
submission will be the only one to do searching against variable enrollment set sizes. Total enrollment and search times
were slightly longer when accounting for submissions that failed in the middle of enrollment or matching and had to be
fixed and restarted (Subsection 4.4).
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
10
4
4.1
FPVTE – F INGERPRINT M ATCHING
Experiment and Test Protocol
API Overview
The FpVTE API consisted of three major steps: feature extraction, finalization, and identification. During all steps of
the evaluation, participants had read-only access to a configuration directory, where configuration files, models, or other
algorithm-specific data could be stored. Not all participants made use of the configuration directory. It was made available as a convenience to those participants who did not wish to store customized traits about their software inside the
submission itself. In practice, many participants were able to perform changes, such as adjusting identification speed,
simply by changing a configuration file within this directory instead of needing to recompile their submission.
4.1.1
Feature Extraction
The first step in the FpVTE evaluation pipeline was feature extraction. During this step, software submissions were given
the opportunity to turn one or more images of fingerprints into a single fingerprint template. There were no requirements
on the format of the fingerprint template, as it was expected that the majority of participants would make use of proprietary template formats. The FpVTE API provided a way to distinguish between the images that would be used for
searching and the images used to compose the enrollment set, though participants were free to treat all images in the same
fashion.
For each instance of feature extraction, the FpVTE test driver called an initialization method a single time, giving the
submission an opportunity to load information that might be needed for feature extraction. After the initialization call,
the feature extraction method was called N times. The FpVTE API provided the finger position, impression type, NFIQ
value (no value provided for slap images), image dimensions, and raw image bytes for each input image during the call to
extract features. As output, participants were to return the finger position, an image quality value, a fingerprint template,
and the size of the fingerprint template as it would be stored in RAM. The RAM size was important to determine the
number of compute nodes needed during identification, because the size of the template on disk might be significantly
larger or smaller than the size of the template in RAM. Participants could optionally return a core and delta coordinate for
the image.
4.1.2
Finalization
As fingerprint templates were returned from feature extraction, the FpVTE test driver added them to a RecordStore (a
key-value pair storage mechanism of the Biometric Evaluation Framework). This allowed I/O operations to be excluded
when calculating the runtime of feature extraction, as well as to allow NIST to store the large quantity of template data in
the most efficient way on NIST’s hardware. Once all features were extracted from enrollment set fingerprint images, the
FpVTE test driver called an API method to “finalize” the enrollment set. Based on the sum of the sizes of templates in RAM,
NIST calculated the appropriate number of compute nodes and passed this information, along with the RecordStore of all
templates, to the finalization method. During this method, the submission was to divide the enrollment set into a partition
for each compute node, as well as perform any sort of indexing, statistics, or other pre-identification tasks required.
Unlike how NIST controlled the storage mechanism for templates in feature extraction, participants had full control of
how to store their finalized enrollment sets. In many cases, the size of the finalized enrollment set was either much larger
or much smaller than the sum of the fingerprint templates sizes returned from feature extraction as stored on disk.
4.1.3
Identification
Once the finalized set was created, its read-only root directory was provided to the identification methods. Searching
the finalized set for candidates occurred over two stages. The first stage was performed on separate compute nodes (if
necessary, as determined by the amount of RAM needed), potentially searching subsets of the enrollment set. Output from
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
11
this first stage was coalesced and provided to the second stage, which took place on a single compute node. These two
stages of identification are described at length in Section 5.
4.2
Test Constraints
In order to complete the evaluation in a reasonable amount of time, as well as to mimic potential operational requirements,
certain constraints were placed on FpVTE submissions. All times are maximum averages over the pre-evaluation datasets
(Section 3), though NIST employed reasonable “cutoff” times to prevent processes from never completing. See Caveats
for additional information about timing.
. Operating System and Compilation Environment: All submitted implementations required 64-bit linkage and were
tested on CentOS 6.2 (kernel version 2.6.32-220.7.1.el6.x86_64). NIST linked participant submissions to the FpVTE
test driver written in C++ with GNU g++ 4.4.6-3, using glibcxx 3.4.13 and glibc 2.12.
. Evaluation Hardware: Timing computations were performed on a Dell M610 with two Intel X5690 3.47 GHz processors and 192 GB of RAM.
. Threaded Computations: Threaded computations were only allowed for the finalization step. All other functions
were not to perform multithreaded computations, as the FpVTE test driver handled parallelism efficiently on the
NIST compute nodes.
. Feature Extraction Time: Feature extraction was required to complete in 3 seconds or less for each input fingerprint.
A four-finger slap was counted as four input fingerprints.
. Finalization Time: Finalization of the enrollment set of fingerprint templates was required to complete in 12 hours
or less on a single compute node.
. Search Time: Search time is the combined time measurement for stage one and stage two identification (not including initialization times). For Class A data, searches needed to complete in under 500 seconds, though implementations that searched in under 20 seconds were reported separately. For Class B and Class C, all searches needed to
complete in under 90 seconds.
4.3
Biometric Evaluation Framework
The FpVTE test driver made use of several C++ classes that are part of the NIST Image Group’s Biometric Evaluation
Framework [11] designed to make writing code for running biometric evaluations easier and more efficient, especially
on NIST-owned equipment. Classes from the framework used in the FpVTE API included key-value pair file storage,
safe dynamic arrays, error handling, and more. While the framework was mainly provided to participants as a means of
interacting with the FpVTE API, many participants chose to use some classes internally in their submissions.
In order to process the massive amount of data required for the evaluation, the submission was executed as a scalable
parallel job. Within the NIST testbed, an implementation of the Message Passing Interface (MPI) [10] was used to execute
the evaluation test programs. By using the MPI software, the size of the parallel job (in terms of participating compute
nodes) can be matched to the size of the input dataset.
The framework supports parallelism by abstracting and hiding the lower-level communication, error handling, and other
facets of the MPI library. The framework application need only implement a few functions in order to become an MPI
parallel job. One feature of the framework is a set of classes that support the distribution of record keys, or key-value
pairs, across the computation cluster. Key distribution allows for driving the test where all nodes have local access to the
data. Key-value distribution has the advantage of running the job with the data source present only on a single node.
Configuration of the job is managed with a simple text file that specifies the input data source, logging system (either files
or a log server), and the number of data consumers assigned to each compute node. By using a configuration file, test
scripts were simpler to invoke, the probability of error was reduced, and replication of the test scenario was implicitly
provided.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
12
4.4
FPVTE – F INGERPRINT M ATCHING
Validation
Participants were provided imagery from publicly available datasets in the same format as the full evaluation datasets.
These validation datasets were used to test implementation functionality before NIST performed the evaluation on a much
larger dataset. Participants ran a validation test with the provided data on their own systems and provided NIST with the
resulting candidate lists and scores from a set of pre-selected searches. NIST did the same on the machines used for the
full evaluation to confirm that both NIST and the participant produced the same results. Validation also provided a way
to make sure that participants were following the conventions of the evaluation, such as returning appropriate quality
values and numbers of candidates, among others.
Care was taken to make the validation process as simple as possible, to let the participant focus more on their submission
than about intricacies specific to FpVTE. To assist, NIST provided a minimal version of the FpVTE test driver software
used in the evaluation to aid participants through the expected calling structure of the evaluation API. Scripts were provided to compile the test driver and run the validation test. All a participant needed to do was create a properly named
software submission, run the script, and submit the generated results file to NIST. Additionally, a build of the Biometric
Evaluation Framework was provided, and many participants made use of features found in the framework in their API
implementations.
4.5
Pre-Evaluation
A timing test was performed to make sure no submission was in violation of the required time limits. First, a sample of the
evaluation dataset was enrolled using ten processes per compute node and the timing of those enrollments were evaluated
to make sure they did not exceed the average of three seconds per fingerprint. I/O time was not included during this step,
as all data was passed to and from the FpVTE test driver in memory. If the submission passed the enrollment timing test,
the full set of data was enrolled and finalized for matching. After enrollment, the next timing test was a random sample
of mated and nonmated searches matched against the maximum enrollment set for each evaluation class. If the average
search time on this sample test set, using a single process on each compute node, completed under the maximum time
limit, the full search set of 30 000 searches was performed. The formula used to compute the total search time was:
Ts = (t1 × b1 ) + t2
(1)
where t1 = average identification stage one time
b1 = number of blades required for identification stage one
t2 = identification stage two time
This formula allowed for a fair comparison between submissions that used one compute node for identification stage one
versus others that may have used two or more compute nodes to store the enrollment set. The formula was vetted with
the participants because it assumes that the enrollment set is evenly distributed across all identification stage one blades
and that search time is fairly linear versus enrollment set size.
Every attempt was made to use the same hardware and number of processes for every submission when reporting timing
results. Additionally, the hardware used for timing was dedicated to the FpVTE test driver process with no other jobs
running on the system except OS related processes.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
4.6
13
Receiving Submissions
Every FpVTE submission received by NIST went through the FpVTE Validation and Pre-Evaluation processes before committing resources to complete the full evaluation. Figure 7 visualizes the submission process with a flowchart. Many
submissions underwent the process multiple times due to defects in the initial submission. Potential FpVTE software
submission defects that would require a resubmission include:
. submission incorrectly signed or encrypted;
. participant’s validation results differ from NIST’s;
. software errors during evaluation;
. maximum average time limit exceeded;
. invalid API implementation.
NIST provided no upper bound on the number of times that a participant could resubmit software in the event of a defect.
Unfortunately, this created an unpredictably large amount of additional work for NIST, through managing submissions,
helping participants debug, reporting status, and the like. Figure 6 details the number of non-debugging submissions
received from participants over the course of the evaluation. The general trend of participants not achieving a “valid”
submission on the first or second attempt ultimately played a part in forcing NIST to extend the original evaluation
deadlines.
Class A
Class B
Class C
Number of Submissions
30
20
10
0
C D E F GH I J K L MO P Q S T U V C D E F GH I J K L MO P Q S T U V C D E F GH I J K L MOP Q S T U V
Participant
Round 1
Round 2
Round 3
Figure 6: Total number of non-debugging submissions received before the final deadline.
At the conclusion of the evaluation, 733 FpVTE validation submissions were received by NIST, including validation
submissions from participants who ultimately withdrew from the evaluation. For this total, a submission is considered
a discrete transmission of non-debugging software to NIST that required action by a NIST employee, on a per-class basis.
There were three rounds during which participants could make submissions. Participants were given an opportunity to
update their submissions at the end of the first two submission rounds. Because FpVTE was a black-box evaluation, NIST
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
14
FPVTE – F INGERPRINT M ATCHING
ultimately had no idea what changes to the submissions were made. A number of submissions majorly regressed in their
lack of defects, so it can be anecdotally inferred that large portions or even entire submissions were changed by some
participants.
NIST imposed certain timing requirements on participants to ensure that the evaluation could be completed on NIST
hardware in a reasonable amount of time, as well as to mimic a possible operational constraint. Several participants tried
to tune their submissions to make use of the maximum amount of time, giving themselves a potential increase in overall
accuracy. While the back-and-forth between NIST and the participants did increase the runtime of the evaluation, timing
defects were the most understandable defect encountered in the evaluation and was primarily caused by NIST-based
evaluation restrictions, not participant error.
During the final submission, participants C, D, E, G, H, I, O, P, Q, and V provided one or more submissions that validated
and ran to completion without NIST encountering defects other than timing restrictions. Participants C, E, G, I, O, and
V met this requirement for all classes of participation in the final submission. Participant E was the only participant to
achieve no defects other than timing restrictions, for every class during all three submission periods.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
15
Validation
Pre-Evaluation
NIST provides sample input
data and evaluation source code.
NIST runs the full evaluation
over a set of 2 000 images
from the evaluation dataset.
Good
Participants run their submissions linked to NIST source
code over provided input data.
NIST generates all templates for evaluation dataset.
Crash
Good
NIST runs black-box submission
over the same sample input data
in the real evaluation environment.
Perform timing test
with subset of searches.
Too Slow
Full Evaluation
Crash
Good
Bad
Participants send blackbox submission and output
from sample data to NIST.
Run remaining (30 000) searches.
Good
Check
results.
Good
Repeat searches for variable enrollment set sizes.
Report results.
Figure 7: Evaluation workflow.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
16
5
FPVTE – F INGERPRINT M ATCHING
Two-Stage Matching
The one-to-many identification step of FpVTE was divided into two distinct phases, under the expectation that storing
the entire enrollment set in RAM on a single compute node would not have been possible. While this was true for many
submissions, it did not hold true for all, especially with Class A data. Even if the submission’s enrollment set fit in the
RAM of a single compute node, the two-stage match technique was still used.
5.1
Enrollment Set Partitioning
NIST determined the amount of RAM needed to hold the entire enrollment set based on a size value returned per finger
during enrollment. In some cases, unique forms of compression were employed, which allowed submissions to use significantly less RAM during identification than reported during enrollment. In other cases, submissions reported very small
RAM requirements per finger, but underlying implementation details required significantly more RAM. This information
was volunteered by participants and documented in Section 9.
After the amount of required RAM was determined, NIST identified the number of compute nodes necessary to support
the RAM requirements. Compute nodes had 192 GB of RAM each (see Subsection 4.2 for more information). Submissions
were then invoked with a method asking them to “finalize” the set of enrollment templates for B compute nodes with a
maximum of 192 GB of RAM per node. This gave submissions an opportunity to partition or subdivide the enrollment set
into more manageable pieces on a per-compute node basis.
5.2
Identification — Stage One
Once the enrollment set had been partitioned, or “finalized,” the next step was to perform searches on each of the partitions. First, the submission’s identification initialization method was called before the first stage of matching. It was
expected that submissions would iterate over their enrollment set partition and load it into the RAM of the compute node
for faster access. Some submissions spent much longer than anticipated in this initialization method, and may have performed some additional binning or pruning that wasn’t otherwise executed during the partitioning step. It’s important to
note that the NIST evaluation compute nodes did not have swap enabled, and so the only memory that could have been
allocated was physical RAM.
After initialization, each search template was matched on each compute node specified during partitioning. To speed up
this process, the FpVTE test driver forked into multiple processes. Under Linux, fork is implemented using copy-on-write
pages, so as long as the submission’s child processes did not write to the RAM allocated during initialization, multiple
identification processes could run in parallel with access to the enrollment set in RAM without fear of RAM being over
allocated. Submissions were allocated a 4 GB RAM disk file system per compute node, where free-form data could be
written. A RAM disk was used to avoid timing I/O as part of the identification process. The FpVTE test driver later
persisted the data to a permanent storage device, and provided this data to the submission during the second stage of
identification.
If an submission failed to perform identification for any reason in the first stage of identification, it was marked as a
miss in stage two, which increased false negative identification rate but slightly decreased false positive identification rate
(Section 6).
5.3
Identification — Stage Two
After all compute nodes had finished the first stage of identification, the second stage was invoked. The submission’s
initialization method for the second stage was called, pointing the submission to the location of its partitioned enrollment
sets (Subsection 5.1) and the RAM disk data from identification stage one. Then, each search template was submitted to
stage two for final identification matching. This search method was to return a candidate list, not exceeding 100 candidates,
with corresponding similarity scores in descending order (where the candidate at rank 1 was the most similar). Each call
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
17
to perform a search included a path to a directory rooted on a RAM disk file system, in which the FpVTE test driver placed
the submission’s output from stage one (Subsection 5.2).
There was no intended control on how the second stage of identification reached the final candidate list, but the search time
limit imposed on submissions combined both stage one and stage two search times (see Section 4 for detailed information
on timing constraints and calculations). Stage two could have been as simple as a sort of stage one results, or as complex
as an additional level of matching involving templates. Based on the timing figures observed during the evaluation, both
trivial and complex stage two implementations were used by FpVTE participants.
Stage two took place on an individual compute node, though the pool of search templates may have been partitioned and
searched on multiple compute nodes independently by the FpVTE test driver to increase throughput.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
18
FPVTE – F INGERPRINT M ATCHING
6
Metrics
The detection error tradeoff (DET) characteristic curve [9] was the primary metric used for comparing accuracy in FpVTE.
Specific points along the DET were examined for making comparisons between submissions. For this initial results report,
the number of mate searches was limited to 10 000 and the number of nonmate searches was limited to 20 000 subjects,
restraining the smallest error rate that could be used with statistical confidence to 10−3 .
In addition to the accuracy of the submission, this report compares speed and computational resource usage. Comparisons
between all submission are made and reported.
6.1
Accuracy
Open-set identification algorithms can make two types of recognition error:
. Search of a biometric sample of an individual not enrolled in the biometric enrollment set (a nonmated search) returns
the biometric reference identifier(s) attributable to one or more enrolled person. This is considered Type I, or false
alarm, because it returns a false identity.
. Search of a biometric sample of an enrolled individual (a mated search) returns an incorrect enrolled identity. This is
considered Type II, or miss, because it misses the correct identity.
FpVTE quantified the accuracy of the open-set identification algorithms as follows:
. False positive identification rate (FPIR), or Type I error rate, is the fraction of the nonmated searches where one or
more enrolled identities are returned at or above threshold (T ) [4]. FPIR is a function of: the size of the enrollment
set (N ), length of candidate lists (L), and score threshold (T ). In the general case, this can be summarized as
FPIR(N, T, L) =
Number of searches with any nonmates returned
above threshold T on candidate list length L
(2)
Number of nonmated searches conducted
and more precisely notated for this evaluation as
PQ
FPIR(T ) =
q=1
H(dq1 − T )
Q
(3)
where Q is the number of searches performed for which there exists no mate in the enrollment set, dq1 is the highest
similarity score reported by the algorithm for the q-th search. The function H(x) is the Heaviside step function
H(x) =
(
0, if x < 0
1, if x ≥ 0
(4)
. False negative identification rate (FNIR), or Type II error rate, is the fraction of the mated searches where the enrolled mate is outside the top R rank or comparison score is below threshold (T ). FNIR is a function of: the size
of the enrollment set (N ), length of candidate lists (L), score threshold (T ), and the number of top candidates being
considered (R). This is summarized in the general case as
FNIR(N, R, T, L) =
Number of mates outside top R ranks or below
threshold T on candidate list length L
Number of mated searches conducted
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
(5)
FPVTE – F INGERPRINT M ATCHING
19
and is defined formally for this evaluation as
FNIR(T ) = 1 −
P
L
1 XX
Ipr [1 − H(dpr − T )]
P p=1 r=1
(6)
where P is the number of searches performed for which there exists a mate in the enrollment set, dpr is the r-th
lowest similarity score reported by the algorithm for the p-th search, and Ipr is 1 if the identity of the r-th candidate
is the same as the identity of search p, or 0 otherwise.
Note that FNIR computation does not care about the cause of a miss: failure to correctly identify a sample (e.g., due to
poor quality), failure to extract a template, failure to generate a comparison score, and software crashes are all dealt with
similarly.
The terms “hit rate,” “reliability,” and “sensitivity” that have been mentioned in some literature on automated fingerprint
identification systems (AFIS) [8, 16] are just the complement of FNIR, computed as 1 − FNIR.
Another widely used accuracy metric is cumulative match characteristic (CMC), which is the fraction of the mated searches
where the enrolled mate is at rank R or better, regardless of its comparison score. CMC is a special case of FNIR, or more
precisely, hit rate, when the constraint on threshold is removed, as shown in Equation 7.
CMC(N, L, R) = 1 − FNIR(N, L, T = 0, R)
(7)
Rank-one hit rate, CMC(N, L, R = 1), is the most common accuracy metric reported in academic and AFIS-related literature. While CMC is reported for the tested submissions, it is an inadequate accuracy metric because its makes strong or
weak hits indistinguishable by ignoring similarity scores, and does not report Type I errors.
6.2
DET Plots
DET characteristic curves are the primary accuracy metric for offline testing of biometric recognition algorithms. Each
point on a DET curve exhibits the false positive identification and false negative identification rates associated with a
certain threshold value. The DET curve spans the entire range of possible threshold values, which is normally the range
of the comparison scores. To reveal the dependence of FNIR and FPIR at a fixed threshold, the DET curves are connected
at points where FNIRs and FPIRs are observed at the same threshold values.
As it is conventional, DET curves are presented for FpVTE submission. In a DET curve, Type I error rates are plotted on
the x-axis and Type II error rates are plotted on the y-axis, giving uniform treatment to both types of error. Both axes
use a logarithmic scale, which spreads out the plot and better distinguishes different well-performing systems. When
calculating FPIR and FNIR, all ranks were considered (L = R = 100).
6.3
Failure to Extract or Match a Template
Failure to extract is the fraction of images for which a template is not generated. Template generation can fail for the
enrollment sample or the search sample. In both cases, failure to extract a template is included as a miss in the computation
of FNIR (see section 6.1).
Additionally, recognition algorithms fail to execute one-to-many searches to produce comparison scores. The result is that
a valid candidate list is not produced. Such failures might be voluntary (e.g., refusal to process a poor quality image) or
involuntary (e.g., software crashes). Either way, it is an undesirable behavior, and should be included in computation of
recognition errors, particularly to allow for fair comparison of submissions. FpVTE treated such failure cases as a miss
and added them to the Type II errors.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
20
FPVTE – F INGERPRINT M ATCHING
Right Hand
Left Hand
1
2
3
4
5
11
13
6
7
8
9
10
12
14
Table 6: Table showing which finger positions were swapped when
the images were flipped for nonmate
searches.
6.4
Figure 9: Example of flipped single-index finger
image.
Computational Efficiency
Another aspect of performance is the computational resources required by a submission. This report includes a comparison of template generation times, one-to-many search times, and template sizes for the submissions, along with their
accuracy at a set FPIR point of 10−3 . The timing numbers are based on data samples for which all submissions were run
under identical conditions on the same hardware.
Ground Truth Errors
The nonmate search errors were resolved by flipping fingerprint images on the
vertical axis for nonmated searches. In order to have the correct topography of the
finger, left/right hand labelings and finger positions were reversed after flipping.
Table 6 shows how the finger positions were swapped to keep the correct finger
topography. Example images are shown in Figure 9, Figure 10, and Figure 11.
0
There were two types of ground truth errors that had to be resolved for FpVTE
datasets. The first involved nonmate searches that had an unknown mate in the
enrollment set. These errors would wrongly increase FPIR (see Section 6). The
second involved mated searches that did not match the presumed mate or had
other unknown mates in the enrollment set. Either of these mated search errors
could wrongly increase FNIR. An example of the effect on a DET curve is shown
in Figure 8. Once the errors show up at a certain threshold, the error rate sharply
increases and remains erroneously high.
FNIR
1
6.5
0
FPIR
1
Figure 8: Example of a DET showing a spike in False Positive Identification Rate. This is usually a sign
of ground truth errors in nonmate
searches.
The mated search errors had to be resolved through manual inspection. A large cause of the unknown mates resulted
from FpVTE using data from multiple sources. These ground truth errors had to be detected by examining the unexpected
high-scoring alleged nonmates produced by the submissions. The results from the submissions were grouped together
to determine which unexpected high-scoring alleged nonmates needed manual inspection. The first step was to look at
searches where all the submissions had an unexpected high-scoring alleged nonmate above a certain threshold. Next,
cases were examined for which only some of the submissions had a high-scoring alleged nonmate. After examining these
cases, if the majority were true mates, the thresholds used were decreased and the process repeated until very few or no
more true mates were found. Not all submissions were used for this process as some produced results that would have
required too much manual work to inspect all of the potential errors produced. This process was repeated for low scoring
alleged mates. The mate searches with low scores were examined to determine if the alleged mate was truly a mate or if
there was a ground truth error. Very low quality images were not removed from the datasets.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Figure 10: Example of flipped left slap image. Note that the flipped image was used as a right slap.
Figure 11: Example of flipped right slap image. Note that the flipped image was used as a left slap.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
21
22
7
FPVTE – F INGERPRINT M ATCHING
Accuracy Results
Subsections 7.1 through 3 show the identification accuracy results, from round 3 submissions, for all three classes of participation. The sections include plots sorted by rank for FNIR at a fixed FPIR of 10−3 , Detection Error Tradeoff (DET) curves
showing accuracy over a range of threshold values, Cumulative Match Characteristic (CMC) curves showing accuracy
over a range of candidate list ranks, and tables with FNIR values at a fixed FPIR. A complete set of full-size DET curves
for each participant are included in Appendix A. Appendix B shows DET and CMC plots for all submissions and classes
grouped on a single page.
This section (7) is followed by sections showing accuracy tradeoff results. Section 8 shows FNIR compared to search
time statistics. Appendix C and Appendix D have a complete set of tables for search time statistics. Appendix E shows
the progression of timing and results from the second to third round of submissions. Section 9 shows FNIR compared
to computational resources such as RAM usage and enrollment (i.e., template creation) times. Appendices F to H have
detailed tables on templates sizes and creation times.
These results are coalesced in Section 10, with tables combining ranked results for each category (FNIR, search time, RAM,
enrollment time). Appendix I has the complete set of tables for ranked results in all classes of participation. Appendix K
plots relative comparisons of FNIR, RAM usage, template creation times, and search times.
Section 11 combines results across classes of participation showing how accuracy varied based on the number of fingers
available for searching. Appendix J plots relative comparisons, by class, for each search set used in FpVTE.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
7.1
Class A
7.1.1
Single-Index Finger Identification
23
The accuracy of single-finger identification is shown with rank-sorted FNIR points (based on right index finger results) in
Figure 12, DET curves in Figure 13, CMC plots in Figure 14, and tables of FNIR points at a fixed FPIR in Tables 7 through
8.
Some observations for Class A single-finger identification include (all FNIR values are at FPIR =10−3 ):
. The most accurate submissions were D, Q, I, V and L2.
. The right index finger was more accurate than the left index finger.
. The most accurate submission D achieved a FNIR of 1.97% for the left index finger and 1.9% for the right index
finger searched against an enrollment set of 100 000 subjects.
. There is a measurable accuracy gap between the top performers and the next level of performers.
. The CMC plots in Figure 14 are not as flat as the other classes. This indicates that while most mates appear within
the top three candidates on the list, there are some that appear further down the list when using single-finger identification.
●
●
0.20
●
●
FNIR @ FPIR =10−3
●
●
●
0.10
●
●
●
●
●
0.05
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0.02
●
●
●
●
●
●
●
●
●
●
●
●
●
●
D1 D2 I2 Q2 I1 Q1 V2 V1 L2 S1 S2 L1 T2 E2 E1 J2 O2 K1 K2 J1 O1 F2 G2 G1 F1 U1 U2 P2 C2 C1 P1 H1 H2 T1 M2 M1
Submission
●
Left Index
●
Right Index
Figure 12: Rank-sorted FNIR @ FPIR = 10−3 for Class A — Single Index Finger searching 30 000 subjects against 100 000
subjects. Submissions “1” and “2” from round 3.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
24
0.5000
0.2000
●
●
●
●●
● ●
●
●
●
I
C
●
●●
●
●
●
P
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
J
D
Left Index (1)
●
●●
●
●
●
●
●
Q
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
E
K
S
●
●
Left Index (2)
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
T
L
●
●
F
Right Index (1)
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
● ●●
●
●●
●●
●
●
●
●
●
●
●
●
G
●
M
U
●
Right Index (2)
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●●
0.5000
●
●
● ●
0.1000
0.0500
0.0200
0.0100
●
0.0050
●
●
0.5000
●
●
●
●
0.0050
0.0100
0.0200
● ●
●
●
0.0500
0.1000
0.2000
●
● ●
●
●
●
●●●●
0.0005
0.0010
0.0020
● ●
●
● ●
●
●●●●
0.0500
0.1000
0.2000
0.2000
●
● ●●
0.0050
0.0100
0.0200
●
●
●●
●●
●
●
0.0050
0.0100
0.0200
0.5000
False Positive Identification Rate
0.0005
0.0010
0.0020
0.1000
●
0.0500
0.1000
0.2000
0.0500
0.0200
0.0100
0.0050
0.0005
0.0010
0.0020
0.5000
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
H
O
●
●
V
●●
●●
●
●
●
●
●
●
●
●
Figure 13: DET for Class A — Single Index Finger searching 30 000 subjects against 100 000 subjects. Submissions “1” and “2” from round 3.
False Negative Identification Rate
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
Miss Rate
5
10
P
I
C
15
Q
J
D
5
10
S
K
E
Left Index (2)
15
Rank
T
L
F
Right Index (1)
5
10
U
M
G
Right Index (2)
15
V
O
H
Figure 14: CMC for Class A — Single Index Finger searching 30 000 subjects against 100 000 subjects. Submissions “1” and “2” from round 3.
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
Left Index (1)
FPVTE – F INGERPRINT M ATCHING
25
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
26
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
K
L
M
O
P
Q
S
T
U
V
Sub. #
FNIR @ FPIR = 10−3
1
30
2
31
0.1337
1
1
0.0197
2
1
0.0197
1
16
0.0745
2
15
0.0723
1
25
0.1111
2
22
0.1082
1
24
0.1089
2
23
0.1086
1
32
0.1576
2
33
0.1607
1
7
0.0257
2
8
0.0278
1
18
0.0786
2
14
0.0712
1
21
0.0883
2
20
0.0875
1
11
0.0625
2
9
0.0351
1
35
0.2995
2
34
0.2921
1
19
0.0818
2
17
0.0766
1
29
0.1308
2
28
0.1272
1
3
0.0222
2
4
0.0226
1
10
0.0571
2
12
0.0650
1
36
NA
2
13
0.0685
1
27
0.1218
2
26
0.1178
1
6
0.0253
2
5
0.0252
0.1335
Table 7: Tabulation of results for Class A — Left Index, with an
enrollment set size of 100 000. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier
used to differentiate between the two submissions each participant
could make. FNIR was computed at the score threshold that gave
FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. The number to the left of a
value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink.
Participant
Letter
C
D
E
F
G
H
I
J
K
L
M
O
P
Q
S
T
U
V
Sub. #
FNIR @ FPIR = 10−3
1
30
2
29
0.1124
1
1
0.0190
2
1
0.0190
1
15
0.0630
2
14
0.0624
1
25
0.0933
2
22
0.0903
1
24
0.0910
2
23
0.0909
1
32
0.1230
2
33
0.1249
1
5
0.0215
2
3
0.0214
1
20
0.0708
2
16
0.0643
1
18
0.0682
2
19
0.0685
1
12
0.0505
2
9
0.0295
1
36
0.2615
2
35
0.2526
1
21
0.0776
2
17
0.0675
1
31
0.1133
2
28
0.1100
1
6
0.0218
2
3
0.0214
1
10
0.0442
2
11
0.0503
1
34
0.1929
2
13
0.0562
1
26
0.0996
2
27
0.1007
1
8
0.0223
2
7
0.0222
0.1132
Table 8: Tabulation of results for Class A — Right Index, with an
enrollment set size of 100 000. Letter refers to the participant’s letter
code found on the footer of this page. Sub. # is an identifier used
to differentiate between the two submissions each participant could
make. FNIR was computed at the score threshold that gave FPIR =
10−3 . The number to the left of a value provides the value’s columnwise ranking, with the best performance shaded in green and the
worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
7.1.2
27
Two-Index Finger Identification
The accuracy of two-finger identification is shown with rank-sorted FNIR points in Figure 15, DET curves in Figure 16,
CMC plots in Figure 17, and a table of FNIR points at a fixed FPIR in Table 9.
Some observations for Class A two-finger identification include (all FNIR values are at FPIR =10−3 ):
. The most accurate submissions were Q, V, D, and I.
. The most accurate submission Q achieved a FNIR of 0.27% searched against an enrollment set of 1.6 million subjects.
. The accuracy gap between the top performers and the second tier, while still measurable, was not as large as singlefinger identification.
. Two-finger identification was far superior to single-finger identification and scaled to much larger enrollment sets.
. The CMC plots in Figure 17 flatten out faster than in single-finger identification. In most cases, the mate is within
the top three candidates of the list or doesn’t appear at all. There were a few submissions that extend down to the
top five to ten candidates.
●
●
●
0.050
●
FNIR @ FPIR =10−3
●
0.020
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0.010
●
●
0.005
●
●
●
●
●
●
●
Q1 Q2 V2 D1 D2 I2 V1 I1 L2 J1 J2 L1 S2 E2 E1 O2 O1 S1 K2 G2 P2 U1 U2 K1 T2 C1 P1 C2 F1 F2 G1 H2 H1 M2 M1 T1
Submission
●
Left and Right Index
Figure 15: Rank-sorted FNIR @ FPIR = 10−3 for Class A — Two Index Fingers searching 30 000 subjects against 1 600 000
subjects. Submissions “1” and “2” from round 3.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
28
0.5000
0.2000
0.1000
0.0500
● ●
0.0200
0.0100
0.0050
● ●
●
I
C
● ●
●
●
●
●
● ●
●
● ●
●
●
●
●●
●
●
●
D
J
●
●●
●
●
●
●
●
●●
S
●
K
E
Left and Right Index (1)
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
0.5000
0.0020
0.0010
0.0005
●
●
Q
●
●
●
●
●
●
●
F
●
L
T
●
●
●
●
●
●
●
●
Left and Right Index (2)
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
G
M
U
●
●
●
●
●
●●
●
●
●
0.0500
0.1000
0.2000
●
●
0.0050
0.0100
0.0200
●
●●
0.0005
0.0010
0.0020
P
●
●●
0.0500
0.1000
0.2000
●
●
●●
0.0050
0.0100
0.0200
False Positive Identification Rate
0.0005
0.0010
0.0020
0.5000
●
●
0.0500
0.1000
0.2000
0.2000
0.1000
0.0500
●
0.0050
0.0100
0.0200
● ●
0.0200
0.0100
0.0050
0.0020
0.0010
0.0005
0.0005
0.0010
0.0020
0.5000
●
●
0.5000
●
●
●
● ●
●
●●
●
●
●
H
O
V
●
●
●
●
●
●
●
Figure 16: DET for Class A — Two Index Fingers searching 30 000 subjects against 1 600 000 subjects. Submissions “1” and “2” from round 3.
False Negative Identification Rate
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
Miss Rate
5
10
P
I
C
15
Q
J
D
5
10
S
K
E
15
Rank
T
L
F
Left and Right Index (2)
5
10
U
M
G
15
V
O
H
Figure 17: CMC for Class A — Two Index Fingers searching 30 000 subjects against 1 600 000 subjects. Submissions “1” and “2” from round 3.
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
Left and Right Index (1)
FPVTE – F INGERPRINT M ATCHING
29
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
30
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
K
L
M
O
P
Q
S
T
U
V
Sub. #
FNIR @ FPIR = 10−3
1
26
0.0368
2
28
0.0374
1
4
0.0030
2
4
0.0030
1
15
0.0207
2
14
0.0202
1
29
0.0386
2
30
0.0412
1
31
0.0515
2
20
0.0311
1
33
0.0686
2
32
0.0684
1
8
0.0058
2
4
0.0030
1
10
0.0143
2
10
0.0143
1
24
0.0360
2
19
0.0286
1
12
0.0146
2
9
0.0072
1
35
NA
2
34
NA
1
17
0.0229
2
16
0.0214
1
27
0.0370
2
21
0.0333
1
1
0.0027
2
1
0.0027
1
18
0.0281
2
13
0.0195
1
36
NA
2
25
0.0366
1
22
0.0336
2
23
0.0358
1
7
0.0034
2
3
0.0028
Table 9: Tabulation of results for Class A — Left and Right Index, with an enrollment set size of 1 600 000. Letter refers to the participant’s letter code
found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. FNIR was
computed at the score threshold that gave FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. The
number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
7.2
31
Class B
The accuracy results for Class B, which included four-finger, eight-finger, and ten-finger identification, are shown with
rank-sorted FNIR points (based on ten finger results) in Figure 18, DET curves in Figure 19, CMC plots in Figure 20, and
tables of FNIR points at fixed FPIR in Tables 10 through 13.
Some observations for Class B include (all FNIR values are at FPIR =10−3 ):
. The most accurate submissions with all ten fingers were I, Q, and D, with FNIRs ranging from 0.09% to 0.2%.
. The next level of submissions were V, E, L, J, O, and G, with FNIRs ranging from 0.24% to 0.4%. The separation
between the top performers and next level of performers was noticeably lower as more fingers are used.
. Right slaps (FNIR = 0.45%, I2) were more accurate than left slaps (FNIR = 0.94%, I2).
. Four-finger identification (FNIR = 0.45%) performed worse than two-finger identification (FNIR = 0.27%), as reported in Subsubsection 7.1.2. Two potential causes for this result are that the submissions in Class B had to perform
segmentation as part of the feature extraction process, and possibly a variation in image quality between the datasets.
Further study will be needed to determine the primary reason four-finger slaps performed worse than two index fingers.
. The matching accuracy improved significantly going from four fingers to eight fingers.
. The best submission I2 achieved FNIRs as follows: left slap (0.94%), right slap (0.45%), left and right slaps (0.15%),
and identification slaps (0.09%).
. The CMC plots in Figure 20 generally show very flat responses. This indicates that for most submissions, the mate
is within the top three candidates on the list or the mate doesn’t appear on the list at all.
●
FNIR @ FPIR =10−3
0.050
●
●
●
●
●
●
0.020
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0.005
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0.010
●
●
0.100
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
D2
I1
0.002
●
●
●
●
0.001
●
I2
Q1 Q2 D1 E2 V2 V1 L1
J2
L2 O2 G2 O1 E1
J1 G1 U1 S1 S2 U2 H1 H2 M2 M1 F1 F2 C2 C1
Submission
●
Identification Flats
●
Left and Right Slap
●
Left Slap
●
Right Slap
Figure 18: Rank-sorted FNIR @ FPIR = 10−3 for Class B — Left Slap, Right Slap, Left and Right Slap, and IDFlats searching
30 000 subjects against 3 000 000 subjects. Submissions “1” and “2” from round 3.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
32
0.2000
●
●
●
●
●
●
●
●
0.0020
●
● ●
0.1000
●
●
0.0500
0.0200
0.0100
●
●
●
●
●
M
●
G
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
● ● ●
●
Left Slap (1)
Left Slap (2)
●
●
●
●
●●
● ●
●
●
●
● ●
●●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
Right Slap (1)
Right Slap (2)
C
●
●
●
●
●
●
●
● ●
●
H
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
D
I
S
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
Left and Right Slap (1)
Left and Right Slap (2)
●
●
0.0500
0.0050
0.0020
0.0010
●
●
●
●
●
●
●
●
●
●
●
●
0.0020
●
●
0.5000
0.0005
●
●
●
●
Q
●
●
●
●
●●
●
●
●
●
●
●
●
E
J
●
●
U
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
● ●●
● ● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
● ●
● ●
●
●●
●
●
Identification Flats (1)
Identification Flats (2)
●
●
●●
●
●
● ●
●●
●
● ●
●
● ●
●
●
●
●
0.0020
●
●
●
●
●●
0.0200
●
●
●
0.0500
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
F
L
●
●
●
●
V
●
●
0.0200
O
●
●
0.5000
●
●●
0.2000
●
0.0200
0.2000
●
●
●●
0.0050
0.1000
●
●
●
0.1000
●●
0.0010
●
0.0010
0.0500
●
●
0.0100
0.0200
●●
●
●
●
●
0.0010
●
● ●
0.0005
●
●
●
●
●
●
0.0500
●
0.2000
●
●
●
0.0100
●
0.0050
0.0100
●
●●
0.0100
0.0050
●
●
0.1000
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
0.2000
●
●
0.1000
0.0020
0.0005
False Positive Identification Rate
0.0050
0.0010
0.0005
0.0005
0.5000
Figure 19: DET for Class B — Left Slap, Right Slap, Left and Right Slap, and IDFlats searching 30 000 subjects against 3 000 000 subjects. Submissions “1”
and “2” from round 3.
False Negative Identification Rate
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
Miss Rate
5
10
O
G
M
15
Q
H
C
Right Slap (1)
Right Slap (2)
5
Rank
10
S
I
D
15
Left and Right Slap (1)
Left and Right Slap (2)
U
J
E
Identification Flats (1)
Identification Flats (2)
5
10
V
L
F
15
Figure 20: CMC for Class B — Left Slap, Right Slap, Left and Right Slap, and IDFlats searching 30 000 subjects against 3 000 000 subjects. Submissions “1”
and “2” from round 3.
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
Left Slap (1)
Left Slap (2)
FPVTE – F INGERPRINT M ATCHING
33
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
34
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Sub. #
FNIR @ FPIR = 10−3
1
22
0.0654
2
21
0.0647
1
6
0.0163
2
5
0.0142
1
13
0.0259
2
7
0.0187
1
29
0.1684
2
28
0.1681
1
18
0.0371
2
17
0.0325
1
23
0.0998
2
24
0.1008
1
4
0.0116
2
1
0.0094
1
15
0.0287
2
10
0.0236
1
16
0.0288
2
14
0.0276
1
30
0.1736
2
27
0.1634
1
12
0.0257
2
11
0.0254
1
2
0.0098
2
3
0.0099
1
25
0.1089
2
26
0.1133
1
20
0.0500
2
19
0.0461
1
9
0.0192
2
8
0.0190
Table 10: Tabulation of results for Class B — Left Slap, with an enrollment set size of 3 000 000. Letter refers to the participant’s letter code
found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make.
FNIR was computed at the score threshold that gave FPIR = 10−3 .
The number to the left of a value provides the value’s column-wise
ranking, with the best performance shaded in green and the worst in
pink.
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Sub. #
FNIR @ FPIR = 10−3
1
24
0.0403
2
23
0.0392
1
6
0.0072
2
2
0.0052
1
13
0.0151
2
7
0.0083
1
29
0.1222
2
28
0.1220
1
18
0.0212
2
16
0.0198
1
25
0.0641
2
26
0.0647
1
5
0.0058
2
1
0.0045
1
14
0.0156
2
10
0.0126
1
15
0.0167
2
17
0.0202
1
30
0.1259
2
27
0.1155
1
12
0.0142
2
11
0.0132
1
3
0.0057
2
3
0.0057
1
21
0.0369
2
22
0.0381
1
19
0.0266
2
20
0.0273
1
8
0.0106
2
9
0.0110
Table 11: Tabulation of results for Class B — Right Slap, with an
enrollment set size of 3 000 000. Letter refers to the participant’s letter
code found on the footer of this page. Sub. # is an identifier used
to differentiate between the two submissions each participant could
make. FNIR was computed at the score threshold that gave FPIR =
10−3 . The number to the left of a value provides the value’s columnwise ranking, with the best performance shaded in green and the
worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Sub. #
FNIR @ FPIR = 10−3
1
30
NA
2
29
NA
1
6
0.0031
2
5
0.0024
1
15
0.0063
2
10
0.0049
1
28
0.0910
2
26
0.0901
1
18
0.0106
2
17
0.0084
1
23
0.0349
2
24
0.0361
1
3
0.0022
2
1
0.0015
1
16
0.0068
2
9
0.0047
1
12
0.0054
2
14
0.0062
1
27
0.0904
2
25
0.0882
1
13
0.0057
2
11
0.0051
1
2
0.0021
2
3
0.0022
1
21
0.0160
2
22
0.0190
1
20
0.0139
2
19
0.0124
1
7
0.0036
2
7
0.0036
Table 12: Tabulation of results for Class B — Left and Right Slap, with
an enrollment set size of 3 000 000. Letter refers to the participant’s
letter code found on the footer of this page. Sub. # is an identifier
used to differentiate between the two submissions each participant
could make. FNIR was computed at the score threshold that gave
FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. The number to the left of a
value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink.
35
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Sub. #
FNIR @ FPIR = 10−3
1
30
NA
2
29
NA
1
6
0.0020
2
2
0.0012
1
16
0.0043
2
7
0.0024
1
27
0.0591
2
27
0.0591
1
18
0.0062
2
14
0.0040
1
23
0.0203
2
24
0.0204
1
2
0.0012
2
1
0.0009
1
17
0.0049
2
11
0.0033
1
10
0.0031
2
11
0.0033
1
26
0.0543
2
25
0.0515
1
15
0.0041
2
13
0.0035
1
2
0.0012
2
2
0.0012
1
20
0.0108
2
21
0.0136
1
19
0.0099
2
22
0.0141
1
9
0.0027
2
7
0.0024
Table 13: Tabulation of results for Class B — Identification Flats, with
an enrollment set size of 3 000 000. Letter refers to the participant’s
letter code found on the footer of this page. Sub. # is an identifier
used to differentiate between the two submissions each participant
could make. FNIR was computed at the score threshold that gave
FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. The number to the left of a
value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
36
FPVTE – F INGERPRINT M ATCHING
7.3
Class C
The accuracy results for Class C, which included ten-finger identification for plain and rolled impression types using
scanned ink and livescan data, are shown with rank-sorted FNIR points (based on ten-finger rolled-to-rolled impression
results) in Figure 21, DET curves in Figure 22, CMC plots in Figure 23, and tables of FNIR points at a fixed FPIR in Tables 14
through 16.
Some observations for Class C include (all FNIR values are at FPIR =10−3 ):
. The most accurate submissions were I, Q, D, and V (ten-finger rolled-to-rolled), with FNIRs ranging from 0.1% to
0.19% for ten-finger plain impressions and ten-finger rolled impressions.
. There was not a clear difference between ten-finger plain-to-plain and ten-finger rolled-to-rolled results. This is a bit
of a surprise, as the ten-finger plain data had to be segmented by the submission and the ten-finger rolled data did
not. It also is a surprise after the observations of lower accuracy seen in Class B four-finger slaps versus the Class
A two index finger accuracy (Subsection 7.2). It might be interesting to perform a similar Class C test with only
four-finger slaps to see if the results are similar to Class B four-finger slap results.
. The second level of performers were E2, J, O, and V (ten-finger plain) with FNIRs ranging from 0.25% to 0.50% for
ten-finger plain-to-plain and ten-finger rolled-to-rolled impressions.
. The best performers were able to handle both rolled and plain impression images with little variation in FNIR, there
was a slight decrease when comparing plain-to-rolled impressions.
. Similar to Class B, the Class C CMC plots in Figure 23 are very flat, indicating the mate is within the top two positions
on the candidate list or it is completely missed.
●
●
●
●
●
●
0.200
●
0.100
FNIR @ FPIR =10−3
●
●
●
0.050
0.020
●
0.010
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0.005
●
●
●
●
●
●
●
●
●
●
●
●
●
0.002
●
●
●
●
●
I1
I2
●
●
●
●
●
●
●
●
●
●
●
●
●
Q2 D1 Q1 V1 D2 V2
J2 O2 O1 E2
J1
L2 C2 C1 L1 E1 H2 H1 G2 U2 U1 G1 F1 F2 M2 M1 S1 S2
Submission
●
Ten−Finger Plain−to−Plain
●
Ten−Finger Plain−to−Rolled
●
Ten−Finger Rolled−to−Rolled
Figure 21: Rank-sorted FNIR @ FPIR = 10−3 for Class C — Ten-Finger plain-to-plain, rolled-to-rolled, and plain-to-rolled
searching 30 000 subjects against 5 000 000 subjects. Submissions “1” and “2” from round 3.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
● ●
0.0020
0.0010
0.0005
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
O
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
Q
H
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
I
●
●
●
●
S
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
False Positive Identification Rate
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
U
●
J
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●●
●
●
●
V
L
●
F
●
●
●
●
●
●
●
●
0.5000
0.0050
False Negative Identification Rate
Figure 22: DET for Class C — Ten-Finger plain-to-plain, rolled-to-rolled, and plain-to-rolled searching 30 000 subjects against 5 000 000 subjects. Submissions “1” and “2” from round 3.
●
0.0200
0.0100
0.0050
● ●
0.2000
0.1000
0.0500
0.5000
0.0005
●
0.0010
●
●
●
0.0020
●
0.0050
●
0.0200
●
0.0100
G
0.0500
0.0020
0.0010
0.0005
0.1000
0.0200
0.0100
0.0050
0.2000
●
0.0005
●
0.0010
●
0.0020
●
0.0050
●
0.0100
●
0.0200
E
0.0010
●
0.0500
D
0.1000
C
0.2000
●
0.5000
0.2000
●
0.1000
●
0.0500
0.5000
●
0.0005
●
0.0020
●
0.0100
M
0.0200
0.5000
0.0500
Ten−Finger Plain−to−Rolled (1)
Ten−Finger Plain−to−Rolled (2)
0.1000
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Rolled−to−Rolled (2)
0.2000
Ten−Finger Plain−to−Plain (1)
Ten−Finger Plain−to−Plain (2)
FPVTE – F INGERPRINT M ATCHING
37
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
38
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
5
E
L
F
Ten−Finger Plain−to−Rolled (1)
Ten−Finger Plain−to−Rolled (2)
D
J
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Rolled−to−Rolled (2)
C
I
Ten−Finger Plain−to−Plain (1)
Ten−Finger Plain−to−Plain (2)
M
H
10
G
5
V
15
U
10
S
5
Q
15
O
10
Rank
15
Figure 23: CMC for Class C — Ten-Finger plain-to-plain, rolled-to-rolled, and plain-to-rolled searching 30 000 subjects against 5 000 000 subjects. Submissions “1” and “2” from round 3.
Miss Rate
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Sub. #
FNIR @ FPIR = 10−3
1
30
NA
2
24
0.0711
1
6
0.0015
2
2
0.0011
1
14
0.0088
2
13
0.0048
1
25
0.0734
2
25
0.0734
1
23
0.0368
2
20
0.0276
1
20
0.0276
2
19
0.0275
1
4
0.0013
2
1
0.0010
1
12
0.0047
2
10
0.0027
1
16
0.0102
2
15
0.0095
1
28
0.0934
2
27
0.0826
1
9
0.0025
2
10
0.0027
1
2
0.0011
2
4
0.0013
1
22
0.0311
2
29
0.1680
1
18
0.0163
2
17
0.0155
1
7
0.0024
2
7
0.0024
Table 14: Tabulation of results for Class C — Ten-Finger Plain-toPlain, with an enrollment set size of 5 000 000. Letter refers to the
participant’s letter code found on the footer of this page. Sub. # is
an identifier used to differentiate between the two submissions each
participant could make. FNIR was computed at the score threshold
that gave FPIR = 10−3 . NA indicates that the operations required
to produce the value could not be performed. The number to the left
of a value provides the value’s column-wise ranking, with the best
performance shaded in green and the worst in pink.
39
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Sub. #
FNIR @ FPIR = 10−3
1
16
0.0094
2
15
0.0085
1
4
0.0015
2
7
0.0018
1
18
0.0106
2
12
0.0050
1
25
0.0536
2
25
0.0536
1
24
0.0447
2
21
0.0333
1
20
0.0201
2
19
0.0199
1
1
0.0013
2
2
0.0014
1
13
0.0051
2
9
0.0033
1
17
0.0097
2
14
0.0083
1
28
0.0783
2
27
0.0716
1
11
0.0034
2
9
0.0033
1
5
0.0017
2
2
0.0014
1
29
0.0860
2
30
0.2462
1
23
0.0358
2
22
0.0351
1
5
0.0017
2
8
0.0019
Table 15: Tabulation of results for Class C — Ten-Finger Rolled-toRolled, with an enrollment set size of 5 000 000. Letter refers to the
participant’s letter code found on the footer of this page. Sub. # is
an identifier used to differentiate between the two submissions each
participant could make. FNIR was computed at the score threshold
that gave FPIR = 10−3 . The number to the left of a value provides
the value’s column-wise ranking, with the best performance shaded
in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
40
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Sub. #
FNIR @ FPIR = 10−3
1
17
0.0149
2
18
0.0169
1
6
0.0028
2
3
0.0018
1
16
0.0137
2
12
0.0056
1
27
0.2514
2
27
0.2514
1
24
0.0649
2
23
0.0521
1
20
0.0291
2
19
0.0285
1
2
0.0014
2
1
0.0011
1
13
0.0071
2
7
0.0034
1
15
0.0136
2
14
0.0129
1
30
0.3067
2
27
0.2514
1
10
0.0041
2
8
0.0036
1
4
0.0020
2
5
0.0022
1
25
0.1017
2
26
0.2366
1
22
0.0378
2
21
0.0295
1
11
0.0052
2
9
0.0039
Table 16: Tabulation of results for Class C — Ten-Finger Plain-to-Rolled, with an enrollment set size of 5 000 000. Letter refers to the participant’s letter
code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. FNIR was
computed at the score threshold that gave FPIR = 10−3 . The number to the left of a value provides the value’s column-wise ranking, with the best
performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
8
41
Accuracy/Search Time Tradeoff
This section examines the tradeoff between FNIR and the amount of time each submission needed to perform identification
searches. The only time restriction placed on the submissions were they must complete searches before a maximum time
limit (Class A: 500 seconds, Class B/C: 90 seconds). NIST allowed for and encouraged two submissions per round. The
intent of this decision was to demonstrate the tradeoff between accuracy and speed, but there was no requirement that the
same basic algorithm be used for the intended “fast” and “slow” submission. It was possible that two completely different
algorithmic approaches were used by a participant, which could introduce other factors when comparing accuracy to
search time for a given participant. As such, the two submissions are simply labeled “1” and “2” in this report. The timing
tables shown in this section are based on the detailed timing tables in Appendix C that show the timing for each stage of
identification.
In addition to the accuracy/timing tradeoffs plots and timing tables in this section, Appendix E contains a full set of
tables that show timing changes that occurred between the last two rounds of submissions. Those tables provide more
data to analyze tradeoffs between search time and accuracy. Again, the algorithm could have changed from one round of
submissions to the next, but it is generally assumed that the basic algorithmic approach stayed the same, while “controls”
were tweaked to improve accuracy.
One general observation noticed across classes of participation was that increased search times are not a guarantee of
increased accuracy. This turned out to be inconsistent across the test. In general, most submissions obtain some improvement in accuracy with increased search times, but most gains are modest, and a few cases had no gain or a slight loss in
accuracy. There are other cases where submissions decreased search time yet still improved accuracy.
It might be useful in future evaluations to encourage competition over extremely fast searches and see which submissions
have a search time vs accuracy advantage, as opposed to tweaking for maximum accuracy, as was done in the current
FpVTE protocol.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
42
8.1
FPVTE – F INGERPRINT M ATCHING
Class A
Tabulated comparisons of identification times for index finger identification submissions are shown in Tables 17 and 18.
The search times shown in these tables are from the “Total/One” column in Tables 24 through 29 included in Appendix C.
For reference, the FNIR values from Section 7 are reprinted to the right of the search times. Class A tables were split into
two groups. The first group includes submissions that performed searches on average in less than 20 seconds, and the
second includes those that took, on average, 20 seconds or longer.
The tables were used to create scatter plots showing accuracy, search times, and search template creation times. Those
plots are shown in Figures 24 through 29.
Some observations for Class A identification times include:
. For single index fingers, there was essentially no accuracy improvement observed with larger search times.
. For two index fingers, there was improvement shown by some participants with increased search time. The overall
benefit might depend on the application.
. It is difficult to compare single-finger identification results to two-finger identification results here, as the enrollment
set sizes used were not the same (100 000 and 1.6 million, respectively).
. The number of processes running (one or ten) didn’t appear to have a major effect on throughput for Class A single
index fingers, and only a slight increase in processing time was observed for two-finger searches. See Appendix C
and Appendix D for complete details.
. Tables 50 through 52 in Appendix E and Figures 30 through 35 show differences between the last two rounds of
submissions. Most submissions lowered FNIR for single finger but required longer search times. The results for two
index fingers were not as consistent. Most lowered FNIR, but some had a significant increase in search time for little
gain in FNIR. While it is not known what changes were made between submissions, there is some indication that
high accuracy can be achieved with some of the fast submissions. The absolute best accuracy is achieved by slightly
slower submissions.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
43
Left Index (Less Than 20−Second Searches)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
0.50
●
T1
●
●
FNIR @ FPIR =10−3
M1
M2
0.20
C1
●
0.10
0.05
●
C2
H1
● H2
●
● P2
P1
●
F1 ●
U1
●
G1
●
●
F2
K2 ●● K1
E1
●● O1 ●
●
J1 ● O2
●
J2
L1
E2
●
●
T2
●
S1
●
L2
I1
●
●
V1
0.02
●
●
D1
0.01
5
●
Q2
Q1
10
15
Median Search Time − One Process (seconds)
Figure 24: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 100 000 subjects and median search time (less than 20 seconds) for a
single process for Class A — Left Index. The color of the data point is used to show the search template creation time. The color scale for search template
creation time is at the top of the plot. Median search times are plotted in seconds. The FNIR and median search time data are from Table 17 and search
template creation times can be found in Table 72 in Appendix H.
Left Index (Greater Than or Equal To 20−Second Searches)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
FNIR @ FPIR =10−3
0.50
0.20
●
●
U2
0.10
G2
●
S2
0.05
●
I2
0.02
0.01
●
V2
●
D2
10
20
30
40
50
Median Search Time − One Process (seconds)
60
Figure 25: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 100 000 subjects and median search time (greater than or equal to 20
seconds) for a single process for Class A — Left Index. The color of the data point is used to show the search template creation time. The color scale
for search template creation time is at the top of the plot. Median search times are plotted in seconds. The FNIR and median search time data are from
Table 17 and search template creation times can be found in Table 72 in Appendix H.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
44
FPVTE – F INGERPRINT M ATCHING
Right Index (Less Than 20−Second Searches)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
0.50
FNIR @ FPIR =10−3
●
0.20
●
M1
M2
●
T1
H1
●
●
P1
C1
H2
●
● ●
●
C2
P2
●
F1
● F2
O1
●
● J1
● O2
●
●
J2
E1
●
0.05 L1
0.10
G1 ●
U1
●
K2 ●● K1
●
E2
●
T2
●
S1
●
L2
●
0.02
V1
0.01
5
Q1
●
I1 ●● Q2
●
D1
10
15
Median Search Time − One Process (seconds)
Figure 26: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 100 000 subjects and median search time (less than 20 seconds) for a
single process. The color of the data point is used to show the search template creation time. The color scale for search template creation time is at the
top of the plot. Median search times are plotted in seconds. The FNIR and median search time data are from Table 17 and search template creation times
can be found in Table 73 in Appendix H.
Right Index (Greater Than or Equal To 20−Second Searches)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
FNIR @ FPIR =10−3
0.50
0.20
0.10
●
U2
●
G2
0.05
●
S2
●
0.02
0.01
●
D2
10
I2
20
30
40
50
Median Search Time − One Process (seconds)
●
V2
60
Figure 27: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 100 000 subjects and median search time (greater than or equal to 20
seconds) for a single process for Class A — Right Index. The color of the data point is used to show the search template creation time. The color scale
for search template creation time is at the top of the plot. Median search times are plotted in seconds. The FNIR and median search time data are from
Table 17 and search template creation times can be found in Table 73 in Appendix H.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
45
Left and Right Index (Less Than 20−Second Searches)
0
1
2
3
4
5
0.100
0.050
●
G1
FNIR @ FPIR =10−3
K1
●
●
●
C2
C1
O1 ●
0.020
●
E1
●
●
L1
J1
0.010
●
I1
0.005
●
V1
0.002
0.001
5
10
15
Median Search Time − One Process (seconds)
Figure 28: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 1 600 000 subjects and median search time (less than 20 seconds) for
a single process for Class A — Left and Right Index. The color of the data point is used to show the search template creation time. The color scale for
search template creation time is at the top of the plot. Median search times are plotted in seconds. The FNIR and median search time data are from
Table 18 and search template creation times can be found in Table 74 in Appendix H.
Left and Right Index (Greater Than or Equal To 20−Second Searches)
0
1
2
3
4
5
0.100
●
●
H1 H2
FNIR @ FPIR =10−3
0.050
0.020
F1 ● F2
T2 ● ● ●●
●
P1
P2
S1 ● ● U1
K2
●
O2
U2
●
●
G2
●
E2
●
S2
●
J2
0.010
●
L2
0.005
V2
●
●
D1
0.002
0.001
100
Q2
●
D2
●
●
Q1
●
I2
200
300
400
Median Search Time − One Process (seconds)
500
Figure 29: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 1 600 000 subjects and median search time (greater than or equal to 20
seconds) for a single process for Class A — Left and Right Index. The color of the data point is used to show the search template creation time. The color
scale for search template creation time is at the top of the plot. Median search times are plotted in seconds. The FNIR and median search time data are
from Table 18 and search template creation times can be found in Table 74 in Appendix H.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
46
FPVTE – F INGERPRINT M ATCHING
Left Index (Less Than 20−Second Searches)
0
0.20
+
+
+
−
FNIR @ FPIR =10−3
5
0.10
−
+
−
−
−
−
−
−
−
−
+
+
−
+
−
−
0.05
−
−
−
0.02
C1
C2
D1
E1
E2
F1
F2
G1
H1
H2
I1
J1
J2
L1
L2
O1
O2
P2
Q1
Q2
S1
U1
V1
Submission
Figure 30: Plots showing difference in FNIR @ FPIR = 10−3 searching 30 000 subjects against 1 600 000 subjects and difference in search times, for a
single search process, between round 2 and round 3 submissions for Class A — Left Index. The “+” symbol indicates that FNIR increased from round 2
to round 3 and “-” indicates a decrease in FNIR. The color of the bar shows the change in search time. The color scale for difference in search time is at
the top of the plot and the units are in seconds. The data for the plots are taken from the tables in Appendix E.
Left Index (Greater Than or Equal To 20−Second Searches)
10
20
30
−
FNIR @ FPIR =10−3
0.10
−
0.05
−
+
−
0.02
D2
G2
I2
S2
V2
Submission
Figure 31: Plots showing difference in FNIR @ FPIR = 10−3 searching 30 000 subjects against 1 600 000 subjects and difference in search times, for a
single search process, between round 2 and round 3 submissions for Class A — Left Index. The “+” symbol indicates that FNIR increased from round 2
to round 3 and “-” indicates a decrease in FNIR. The color of the bar shows the change in search time. The color scale for difference in search time is at
the top of the plot and the units are in seconds. The data for the plots are taken from the tables in Appendix E.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
47
Right Index (Less Than 20−Second Searches)
−5
0
5
+
FNIR @ FPIR =10−3
0.20
+
+
+
−
0.10
−
−
+
−
−
−
−
−
−
−
+
−
+
−
0.05
−
−
−
−
Q1
Q2
−
0.02
C1
C2
D1
E1
E2
F1
F2
G1
H1
H2
I1
J1
J2
L1
L2
O1
O2
P2
S1
T1
U1
V1
Submission
Figure 32: Plots showing difference in FNIR @ FPIR = 10−3 searching 30 000 subjects against 1 600 000 subjects and difference in search times, for a
single search process, between round 2 and round 3 submissions for Class A — Right Index. The “+” symbol indicates that FNIR increased from round
2 to round 3 and “-” indicates a decrease in FNIR. The color of the bar shows the change in search time. The color scale for difference in search time is at
the top of the plot and the units are in seconds. The data for the plots are taken from the tables in Appendix E.
Right Index (Greater Than or Equal To 20−Second Searches)
10
20
30
−
FNIR @ FPIR =10−3
0.10
−
0.05
−
−
−
0.02
D2
G2
I2
S2
V2
Submission
Figure 33: Plots showing difference in FNIR @ FPIR = 10−3 searching 30 000 subjects against 1 600 000 subjects and difference in search times, for a
single search process, between round 2 and round 3 submissions for Class A — Right Index. The “+” symbol indicates that FNIR increased from round
2 to round 3 and “-” indicates a decrease in FNIR. The color of the bar shows the change in search time. The color scale for difference in search time is at
the top of the plot and the units are in seconds. The data for the plots are taken from the tables in Appendix E.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
48
FPVTE – F INGERPRINT M ATCHING
Left and Right Index (Less Than 20−Second Searches)
−40
−30
−20
−10
0
−
FNIR @ FPIR =10−3
0.100
0.050
+
+
−
+
0.020
−
0.010
−
−
0.005
C1
C2
G1
I1
J1
L1
O1
V1
Submission
Figure 34: Plots showing difference in FNIR @ FPIR = 10−3 searching 30 000 subjects against 1 600 000 subjects and difference in search times, for a
single search process, between round 2 and round 3 submissions for Class A — Left and Right Index. The “+” symbol indicates that FNIR increased
from round 2 to round 3 and “-” indicates a decrease in FNIR. The color of the bar shows the change in search time. The color scale for difference in
search time is at the top of the plot and the units are in seconds. The data for the plots are taken from the tables in Appendix E.
Left and Right Index (Greater Than or Equal To 20−Second Searches)
0
100
200
300
−
FNIR @ FPIR =10−3
0.200
0.100
−
0.050
−
−
0.020
+
+
+
−
−
−
−
−
+
−
+
0.010
0.005
−
D1
D2
−
−
−
E1
E2
F1
F2
G2
H1
H2
I2
J2
L2
O2
P2
−
−
Q1
Q2
S1
S2
U1
U2
V2
Submission
Figure 35: Plots showing difference in FNIR @ FPIR = 10−3 searching 30 000 subjects against 1 600 000 subjects and difference in search times, for a
single search process, between round 2 and round 3 submissions for Class A — Left and Right Index. The “+” symbol indicates that FNIR increased
from round 2 to round 3 and “-” indicates a decrease in FNIR. The color of the bar shows the change in search time. The color scale for difference in
search time is at the top of the plot and the units are in seconds. The data for the plots are taken from the tables in Appendix E.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
1
0.26
24
2
6
0.76
25
1
20
7.32
1
Time
FNIR
1
0.24
24
0.1337
6
0.69
23
0.1124
0.0197
20
7.46
1
0.0190
0.1335
0.1132
3
0.35
12
0.0745
3
0.32
11
0.0630
2
30
16.90
11
0.0723
29
16.27
10
0.0624
1
11
2.49
20
0.1111
11
2.38
20
0.0933
2
14
3.56
18
0.1082
15
3.60
18
0.0903
1
21
9.74
19
0.1089
24
10.10
19
0.0910
1
15
3.61
26
0.1576
14
3.56
26
0.1230
2
17
4.13
27
0.1607
17
4.05
27
0.1249
1
22
9.94
5
0.0257
22
9.83
3
0.0215
1
4
0.54
14
0.0786
4
0.51
16
0.0708
2
7
1.25
10
0.0712
7
1.13
12
0.0643
1
25
10.44
17
0.0883
26
10.42
14
0.0682
2
23
10.32
16
0.0875
25
10.27
15
0.0685
1
2
0.29
8
0.0625
2
0.26
8
0.0505
2
12
3.32
6
0.0351
13
3.26
6
0.0295
1
10
1.84
29
0.2995
10
1.78
30
0.2615
2
26
10.70
28
0.2921
21
9.39
29
0.2526
1
5
0.62
15
0.0818
5
0.56
17
0.0776
2
9
1.56
13
0.0766
9
1.36
13
0.0675
1
8
1.32
23
0.1308
8
1.24
25
0.1133
2
13
3.33
22
0.1272
12
3.07
22
0.1100
1
28
15.70
2
0.0222
28
14.24
4
0.0218
2
24
10.43
3
0.0226
23
9.98
2
0.0214
1
29
16.86
7
0.0571
30
16.55
7
0.0442
1
16
3.99
30
NA
16
3.73
28
0.1929
2
18
5.97
9
0.0685
19
5.87
9
0.0562
U
1
27
14.42
21
0.1218
27
10.76
21
0.0996
V
1
19
6.01
4
0.0253
18
5.60
5
0.0223
D
2
3
41.84
1
0.0197
2
28.64
1
0.0190
G
2
5
54.03
5
0.1086
5
59.55
5
0.0909
I
2
2
40.99
3
0.0278
3
40.35
2
0.0214
F
G
H
I
< 20 seconds
Right Index
FNIR
1
E
J
K
L
M
O
P
Q
S
T
≥ 20 seconds
Left Index
Time
Sub. #
1
49
S
2
4
44.66
4
0.0650
4
46.05
4
0.0503
U
2
1
24.16
6
0.1178
1
20.10
6
0.1007
V
2
6
65.35
2
0.0252
6
61.24
3
0.0222
Table 17: Tabulation of median identification times for Class A. Submissions were split into two groups. The first group includes submissions that
performed searches on average in less than 20 seconds, and the second includes those that took, on average, 20 seconds or longer. Letter refers to the
participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could
make. The Time column shows the time used to perform a search over an enrollment set of 100 000. Time values are median times reported in seconds,
but were originally recorded to microsecond precision. The FNIR column shows FNIR for each submission at FPIR = 10−3 . NA indicates that the
operations required to produce the value could not be performed. The number to the left of a value provides the value’s column-wise ranking, with the
best performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
50
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
< 20 seconds
C
1
1
2.08
7
0.0368
2
4
6.35
8
0.0374
G
1
3
5.22
9
0.0515
I
1
8
17.87
2
0.0058
J
1
6
13.64
3
0.0143
K
1
9
18.01
6
0.0360
L
1
2
2.19
4
0.0146
O
1
7
14.46
5
0.0229
V
1
5
9.29
1
0.0034
1
15
70.99
4
0.0030
2
23
237.43
4
0.0030
1
1
16.35
11
0.0207
2
27
518.11
10
0.0202
1
13
60.66
21
0.0386
2
16
73.95
22
0.0412
2
22
221.16
15
0.0311
1
8
36.71
24
0.0686
2
12
52.25
23
0.0684
I
2
25
338.88
4
0.0030
J
2
7
33.35
8
0.0143
K
2
6
32.93
14
0.0286
L
2
3
23.42
7
0.0072
1
5
31.44
26
NA
2
20
171.24
25
NA
2
10
43.53
12
0.0214
1
14
63.65
20
0.0370
2
17
101.16
16
0.0333
1
21
212.69
1
0.0027
2
19
161.02
1
0.0027
D
E
F
G
H
≥ 20 seconds
Left and Right Index
Time
FNIR
Sub. #
M
O
P
Q
S
T
U
V
1
2
23.00
13
0.0281
2
26
495.50
9
0.0195
1
4
25.68
27
NA
2
9
37.23
19
0.0366
1
11
45.51
17
0.0336
2
24
240.40
18
0.0358
2
18
127.65
3
0.0028
Table 18: Tabulation of median identification times for Class A. Submissions were split into two groups. The first group includes submissions that
performed searches on average in less than 20 seconds, and the second includes those that took, on average, 20 seconds or longer. Letter refers to the
participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could
make. The Time column shows the time used to perform a search over an enrollment set of 1 600 000. Time values are median times reported in seconds,
but were originally recorded to microsecond precision. The FNIR column shows FNIR for each submission at FPIR = 10−3 . NA indicates that the
operations required to produce the value could not be performed. The number to the left of a value provides the value’s column-wise ranking, with the
best performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
8.2
51
Class B
Tabulated comparisons of identification times for IDFlat submissions are shown in Table 19. The search times shown in
this table are from the “Total/One” column in Tables 30 through 33 included in Appendix C. For reference, the FNIR
values from Section 7 are reprinted to the right of the identification times.
The tables were used to create scatter plots showing accuracy, search times, and search template creation times. Those
plots are shown in Figures 36 through 39.
Some observations for Class B identification times include:
. Most submissions had some improvement in accuracy with longer search times.
. Results vary, but some submissions (D, G, I1, Q1) performed searches faster when more fingers were available, while
others (H, L, S) required longer search times with more fingers.
. Most search times increased modestly when ten processes were running in parallel, compared to the single process
timing test. See Appendix C and Appendix D for complete details.
. Tables 30 through 33 in Appendix E and Figures 40 through 43 show differences between the last two rounds of
submissions. There are a variety of results in these tables. Again, most had improvement in FNIRs, but some with
longer search times and some with dramatically shorter search times. Like Class A, there are certainly indications
that high accuracy can be achieved with some fast submissions.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
52
FPVTE – F INGERPRINT M ATCHING
Left Slap
0
2
4
6
0.2000
8
F1 ● ● M1
C1
0.0500
●
M2
●
H2
U1
●
●
G1
O1
●
●
J1
L2
●
●
E2
V2
●
V1
O2 ● ● J2
D1
●
●
I1
●
0.0100
U2 ●
G2
●
●
L1
●
●
●
S2
●
●
S1
C2
E1
0.0200
14
●
H1 ●
●
12
F2
0.1000
FNIR @ FPIR =10−3
10
D2
Q1 ●●
Q2
●
I2
0.0050
0.0020
0.0010
0.0005
20
40
60
Median Search Time − One Process (seconds)
80
Figure 36: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 3 000 000 subjects and median search time for a single process for Class
B — Left Slap. The color of the data point is used to show the search template creation time. The color scale for search template creation time is at the
top of the plot. Median search times are plotted in seconds. The FNIR and median search time data are from Table 19 and search template creation times
can be found in Table 75 in Appendix H.
Right Slap
0
2
4
6
8
10
12
14
0.2000
M1
●
●
F1
FNIR @ FPIR =10−3
0.1000
0.0500
●
M2
H1 ●
●
C1
0.0200
●
0.0100
●
F2
●
L1
●
●
H2
S2
●●
S1
C2
●
G1
●
U2 ●
G2
●
L2
●
J1
E1
0.0050
●
O1
O2
●
●
J2
V2
●
V1 ●
●
E2
D1
●
●
I2
U1 ●
I1
●
●
Q2
●
●
Q1
D2
0.0020
0.0010
0.0005
20
40
60
Median Search Time − One Process (seconds)
80
Figure 37: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 3 000 000 subjects and median search time for a single process for Class
B — Right Slap. The color of the data point is used to show the search template creation time. The color scale for search template creation time is at the
top of the plot. Median search times are plotted in seconds. The FNIR and median search time data are from Table 19 and search template creation times
can be found in Table 76 in Appendix H.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
●
Left and Right Slap
●
C1
0
53
C2
5
10
15
20
25
0.2000
FNIR @ FPIR =10−3
0.1000
M1 ●● F1
●
●
F2
M2
0.0500
H1 ●
●
H2
S2
●
S1 ●●
U2 ● U1
0.0200
●
0.0100
G2
●
G1
●
E1
0.0050
●
●
●
J1
L2
L1
●
●
E2
●
V2
●
V1
●
0.0020
O2
●
●
J2
O1
●
I1
Q2
D1
●
D2 ●
●
Q1
●
I2
0.0010
0.0005
20
40
60
Median Search Time − One Process (seconds)
80
Figure 38: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 3 000 000 subjects and median search time for a single process for Class
B — Left and Right Slap. The color of the data point is used to show the search template creation time. The color scale for search template creation time is
at the top of the plot. Median search times are plotted in seconds. The FNIR and median search time data are from Table 19 and search template creation
times can be found in Table 77 in Appendix H.
IDFlat
●
C1
0
5
●
C2
10
15
20
0.2000
FNIR @ FPIR =10−3
0.1000
F1
●
●
●
0.0500
F2
M1
●
M2
H1 ●
0.0200
U2 ●
●
U1 ●S1
0.0100
●
0.0050
G1
J1
●
G2
●
●
E1
0.0020
O1
●
●
●
L2
L1
●
V1
E2
●
●
●
V2
O2
●
●
J2
D1
●
0.0010
H2
●
S2
●
I1
Q1 ●
●
D2
●
Q2
●
I2
0.0005
20
40
60
Median Search Time − One Process (seconds)
80
Figure 39: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 3 000 000 subjects and median search time for a single process for Class
B — Identification Flats. The color of the data point is used to show the search template creation time. The color scale for search template creation time is
at the top of the plot. Median search times are plotted in seconds. The FNIR and median search time data are from Table 19 and search template creation
times can be found in Table 78 in Appendix H.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
54
FPVTE – F INGERPRINT M ATCHING
Left Slap
0
20
40
0.20
+
FNIR @ FPIR =10−3
+
+
0.10
−
−
−
−
−
−
−
0.05
−
−
−
−
−
−
0.02
−
−
−
−
−
−
0.01
C1
C2
D1
D2
E1
E2
G1
G2
H1
H2
I1
I2
J1
J2
L1
L2
M2
O1
O2
Q1
Q2
V1
Submission
Figure 40: Plots showing difference in FNIR @ FPIR = 10−3 searching 30 000 subjects against 3 000 000 subjects and difference in search times, for a
single search process, between round 2 and round 3 submissions for Class B — Left Slap. The “+” symbol indicates that FNIR increased from round 2 to
round 3 and “-” indicates a decrease in FNIR. The color of the bar shows the change in search time. The color scale for difference in search time is at the
top of the plot and the units are in seconds. The data for the plots are taken from the tables in Appendix E.
Right Slap
−20
0
20
40
−
0.100
FNIR @ FPIR =10−3
+
0.050
+
−
+
+
−
−
−
−
−
−
−
−
0.020
−
−
0.010
−
−
−
−
−
−
Q1
Q2
0.005
C1
C2
D1
D2
E1
E2
G1
G2
H1
H2
I1
I2
J1
J2
L1
L2
M2
O1
O2
V1
Submission
Figure 41: Plots showing difference in FNIR @ FPIR = 10−3 searching 30 000 subjects against 3 000 000 subjects and difference in search times, for a
single search process, between round 2 and round 3 submissions for Class B — Right Slap. The “+” symbol indicates that FNIR increased from round 2
to round 3 and “-” indicates a decrease in FNIR. The color of the bar shows the change in search time. The color scale for difference in search time is at
the top of the plot and the units are in seconds. The data for the plots are taken from the tables in Appendix E.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
55
Left and Right Slap
−20
0
20
+
0.100
0.050
FNIR @ FPIR =10−3
40
+
+
−
−
−
−
0.020
−
−
0.010
0.005
−
−
−
−
−
−
−
−
−
0.002
D1
D2
E1
E2
G1
G2
H1
H2
I1
I2
J1
J2
L1
L2
M2
O1
O2
+
+
Q1
Q2
V1
Submission
Figure 42: Plots showing difference in FNIR @ FPIR = 10−3 searching 30 000 subjects against 3 000 000 subjects and difference in search times, for a
single search process, between round 2 and round 3 submissions for Class B — Left and Right Slap. The “+” symbol indicates that FNIR increased from
round 2 to round 3 and “-” indicates a decrease in FNIR. The color of the bar shows the change in search time. The color scale for difference in search
time is at the top of the plot and the units are in seconds. The data for the plots are taken from the tables in Appendix E.
Identification Flats
−40
−20
0
20
−
+
FNIR @ FPIR =10−3
0.050
−
0.020
40
−
−
−
−
−
−
0.010
−
−
−
−
−
0.005
−
−
−
0.002
−
+
+
Q1
Q2
0.001
D1
D2
E1
E2
G1
G2
H1
H2
I1
I2
J1
J2
L1
L2
M2
O1
O2
V1
Submission
Figure 43: Plots showing difference in FNIR @ FPIR = 10−3 searching 30 000 subjects against 3 000 000 subjects and difference in search times, for a
single search process, between round 2 and round 3 submissions for Class B — IDFlat. The “+” symbol indicates that FNIR increased from round 2 to
round 3 and “-” indicates a decrease in FNIR. The color of the bar shows the change in search time. The color scale for difference in search time is at the
top of the plot and the units are in seconds. The data for the plots are taken from the tables in Appendix E.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
56
24
0.0072
0.0392
0.0403
20
5
3
61.24
10.70
6.40
6
29
30
0.0031
NA
NA
17
4
3
45.15
10.21
7.92
6
29
30
0.0020
NA
NA
Identification Flats
Time
FNIR
3.67
6
23
Left and Right Slap
Time
FNIR
6.50
FNIR
2
53.32
Right Slap
0.0654
4
Time
22
19
FNIR
3.74
0.0647
Left Slap
2
0.0163
Time
1
6
21
0.0024
6.74
0.0591
52.23
7
0.0591
4
27
0.0062
18
43.61
27
0.0040
2
37.56
18
0.0203
1
15
14
0.0012
12
4.26
49.34
23
0.0043
0.0049
16.26
2
0.0910
1
20
82.66
16
10
0.0901
6
6.76
28
0.0106
26
46.70
26
0.0084
2
33.09
18
0.0349
18
27.27
17
0.0204
0.0024
9
10
3.89
36.36
23
24
0.0012
0.0063
0.0083
34.55
2
5
0.1222
1
12
70.38
86.56
1
15
7
0.1220
11
27
19.07
6.12
29
0.0212
25
7
63.05
34.47
28
0.0198
0.0361
16
0.0049
0.0009
2
31.87
18
0.0641
24
0.0022
17
21
45.92
16
3
43.86
0.0052
8
10
10.13
25
73.06
1
25.38
0.0151
0.0187
37.28
26
37.19
8
2
0.1684
5
15
50.16
13
0.0068
0.0015
13
7
0.1681
13
0.0647
17
16
2.97
29
0.0371
17
26
0.0058
46.13
57.19
34.37
28
0.0325
5
26.15
1
30.09
18
0.0998
52.84
1
7
21
43.59
17
18
55.87
0.0156
0.0045
0.0142
8
10
16.24
23
20
14
0.0259
2
59.49
0.1008
14
41.13
5
1
6
14
49.75
24
0.0116
24.14
13
2
23
4
7
2.82
1
16
51.77
1
0.0287
0.0094
58.10
2
17
55.96
15
1
1
21
38.26
22
2
13
25.61
2
1
7
27
14
0.0057
0.0882
0.0904
0.0062
14
21
11
23
13
9
87.60
66.01
42.92
60.02
35.64
65.91
38.87
28.59
19
21
20
2
2
13
15
25
26
11
0.0027
0.0141
0.0099
0.0136
0.0108
0.0012
0.0012
0.0035
0.0041
0.0515
0.0543
0.0033
1
20.78
25
0.0051
24
88.46
22
0.0024
0.0033
27.15
13
0.0021
29
86.60
9
11
6
45.91
11
0.0022
30
75.28
7
22
8
38.34
2
0.0160
28
34.96
9
0.0202
16
68.46
3
0.0190
25
49.30
24
0.1259
14
65.02
21
0.0139
10
10
17
0.1155
23
53.40
22
0.0124
19
27
30
0.0142
22
83.33
20
0.0036
10
12.66
27
0.0132
19
82.13
19
0.0036
27
32.03
12
0.0057
29
83.62
7
0.0031
6
87.41
11
0.0057
27
82.40
7
10
9
35.61
3
0.0369
30
38.93
60.13
0.0276
29
73.93
3
0.0381
28
52.80
14.50
0.1736
12
64.04
21
0.0266
15
5
14
0.1634
26
61.94
22
0.0273
18
0.0047
30
0.0257
23
70.54
19
0.0106
0.0054
12.37
27
0.0254
22
70.84
20
0.0110
12
31.17
12
0.0098
24
89.26
8
69.28
5
87.21
11
0.0099
25
80.11
9
10.48
1
9
37.01
2
0.1089
30
35.43
4
2
29
73.15
3
0.1133
28
48.51
0.0126
1
12
53.24
25
0.0500
11
0.0167
2
26
54.02
26
0.0461
16
15
1
19
71.69
20
0.0192
5.47
2
20
71.77
19
0.0190
74.09
1
24
80.03
9
3
2
25
89.07
8
0.0236
1
28
36.07
0.0288
2
30
47.97
16
1
11
5.56
2
15
74.38
1
3
2
2
2
1
Sub. #
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Table 19: Tabulation of median identification times for Class B. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between
the two submissions each participant could make. The Time column shows the time used to perform a search over an enrollment set of 3 000 000. Time values are median times reported in
seconds, but were originally recorded to microsecond precision. The FNIR column shows FNIR for each submission at FPIR = 10−3 . NA indicates that the operations required to produce the
value could not be performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
8.3
57
Class C
Tabulated comparisons of identification times for ten-finger identification submissions are shown in Table 20. The search
times shown in this table are from the “Total/One” column in Tables 34 through 36 included in Appendix C. For reference,
FNIR values from Section 7 are reprinted to the right of the identification times.
The tables were used to create scatter plots showing accuracy, search times, and search template creation times. Those
plots are shown in Figure 44.
Some observations for Class C identification times include:
. Like classes A and B, gains varied across the participants, but most had some level of improvement in accuracy with
longer search times.
. Results for some submissions varied between ten-finger plain-to-plain and ten-finger rolled-to-rolled impressions.
Some were faster with plain impressions and others were faster with rolled impressions. The most accurate submissions appeared to match both types accurately.
. Tables 34 through 36 in Appendix E and Figure 45 show differences between the last two rounds of submissions.
Like classes A and B, these results indicate that high accuracy can be achieved with some fast submissions, but the
absolute best accuracy was not the fastest submission.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
58
FPVTE – F INGERPRINT M ATCHING
Plain−to−Plain
●
0
5 C1
10
15
20
25
30
0.2000
●
C2
0.0500
●
M2
●
FNIR @ FPIR =10−3
S2
M1
●
0.1000
●
●
F1
F2
S1
●
●
G1
H2 ●
●
G2
0.0200
●
H1
●
●
U2
0.0100
●
E1
L2
0.0050
●
●
J1
E2
20
●
O1
●
Q2 ●Q1
●
●
I2
0.0010
O2
●
●
V2 ●
●
V1
0.0020
0.0005
U1
●
L1
●
●
J2
D1
●
I1
D2
40
60
Median Search Time − One Process (seconds)
80
Rolled−to−Rolled
0
5
10
15
20
25
30
●
S2
0.2000
FNIR @ FPIR =10−3
0.1000
M1
●
● M2
●
S1
●
0.0500
●
F1
●
G1
F2
U1
●
●
●
U2
G2
0.0200
H1 ● H2
C1
●
●
0.0100
E1
●
●
L2
●
C2
L1
0.0050
●
●
J1
E2
●
J2 ● ● O2
O1
0.0020
0.0010
0.0005
20
●
D1
●
●
V1
●
I2
V2
●
I1
Q1●●● D2
Q2
40
60
Median Search Time − One Process (seconds)
80
Plain−to−Rolled
0
5
10
15
20
M1 ●
M2 ●●
F1
0.2000
25
●
30
F2
S2 ●
S1 ●
0.1000
FNIR @ FPIR =10−3
●
G1
0.0500
●
U1
●
G2
H1 ● ● H2
0.0200
E1
●
L2
●
L1
●
C1
●
E2 ●
0.0050
J1
V1
●
●
Q2
Q1
O1
●
D1
●●
0.0020
●
I2
20
●
O2 ● ● V2
J2
D2
●
●
0.0010
0.0005
U2
C2
●
●
0.0100
●
I1
40
60
Median Search Time − One Process (seconds)
80
Figure 44: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 5 000 000 subjects and median search time for a single process for Class
C. The color of the data point is used to show the search template creation time. The color scale for search template creation time is at the top of the plot.
Median search times are plotted in seconds. The FNIR and median search time data are from Table 20 and search template creation times can be found
in Tables 79 and 80 in Appendix H.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
59
Ten−Finger Plain−to−Plain
−20
0
FNIR @ FPIR =10−3
40
−
0.200
0.100
20
+
+
−
+
0.050
−
−
−
0.020
−
−
0.010
−
−
−
0.005
−
0.002
−
−
−
−
−
−
0.001
C2
D1
D2
E1
E2
F1
F2
G1
G2
H1
H2
I1
I2
J1
J2
L1
L2
Q1
Q2
V1
Submission
Ten−Finger Rolled−to−Rolled
−60
0.100
−
−40
−20
20
40
−
+
0.050
FNIR @ FPIR =10−3
0
+
+
+
−
−
−
0.020
−
−
−
−
0.010
−
0.005
−
−
−
0.002
+
−
C1
C2
D1
D2
E1
E2
F1
F2
G1
G2
H1
H2
I1
I2
J1
J2
L1
L2
Q1
−
Q2
−
V1
Submission
Ten−Finger Plain−to−Rolled
−40
−20
+
+
FNIR @ FPIR =10−3
−
20
−
0.200
0.100
0
−
−
0.050
−
−
−
−
−
0.020
−
−
0.010
−
0.005
−
−
−
+
−
−
−
0.002
0.001
C1
C2
D1
D2
E1
E2
F1
F2
G1
G2
H1
H2
I1
I2
J1
J2
L1
L2
Q1
Q2
V1
Submission
Figure 45: Plots showing difference in FNIR @ FPIR = 10−3 searching 30 000 subjects against 5 000 000 subjects and difference in search times, for a
single search process, between round 2 and round 3 submissions for Class C. The “+” symbol indicates that FNIR increased from round 2 to round 3 and
“-” indicates a decrease in FNIR. The color of the bar shows the change in search time. The color scale for difference in search time is at the top of the
plot and the units are in seconds. The data for the plots are taken from the tables in Appendix E.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
60
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Ten-Finger Plain-to-Plain
Time
FNIR
Sub. #
4
18.30
30
NA
2
5
18.38
24
1
25
73.93
6
2
26
74.36
1
2
12.77
2
17
52.91
1
15
51.69
2
18
1
1
2
1
2
1
Ten-Finger Rolled-to-Rolled
Time
FNIR
4
20.15
16
0.0094
0.0711
6
21.97
15
0.0015
17
58.92
4
2
0.0011
27
74.97
14
0.0088
2
8.63
13
0.0048
11
42.73
12
25
0.0734
16
55.79
25
60.86
25
0.0734
20
68.20
7.71
23
0.0368
1
7.78
7
23.69
20
0.0276
3
21
68.07
20
0.0276
30
20
65.36
19
0.0275
16
52.41
4
0.0013
2
10
38.28
1
1
12
43.57
12
2
28
77.75
1
3
17.51
2
6
1
11
2
1
Ten-Finger Plain-to-Rolled
Time
FNIR
6
33.68
17
0.0149
0.0085
8
34.85
18
0.0169
0.0015
13
41.67
6
0.0028
7
0.0018
16
57.36
3
0.0018
18
0.0106
1
8.99
16
0.0137
0.0050
7
34.06
12
0.0056
0.0536
19
58.57
27
0.2514
25
0.0536
21
61.87
27
0.2514
24
0.0447
2
16.78
24
0.0649
19.46
21
0.0333
4
22.14
23
0.0521
84.51
20
0.0201
22
64.83
20
0.0291
29
84.50
19
0.0199
23
65.74
19
0.0285
15
54.56
1
0.0013
15
52.47
2
0.0014
0.0010
7
30.82
2
0.0014
12
41.48
1
0.0011
0.0047
9
33.02
13
0.0051
9
38.41
13
0.0071
10
0.0027
19
67.98
9
0.0033
25
67.88
7
0.0034
16
0.0102
8
31.68
17
0.0097
5
27.47
15
0.0136
23.53
15
0.0095
5
20.25
14
0.0083
3
20.18
14
0.0129
43.37
28
0.0934
12
48.08
28
0.0783
17
57.63
30
0.3067
13
46.70
27
0.0826
13
48.80
27
0.0716
18
57.91
27
0.2514
23
68.54
9
0.0025
14
54.42
11
0.0034
20
59.05
10
0.0041
2
24
73.41
10
0.0027
21
69.30
9
0.0033
24
66.80
8
0.0036
1
9
36.86
2
0.0011
26
74.40
5
0.0017
11
39.76
4
0.0020
2
8
35.35
4
0.0013
25
74.22
2
0.0014
10
39.37
5
0.0022
1
29
86.56
22
0.0311
22
69.71
29
0.0860
30
88.48
25
0.1017
2
30
91.74
29
0.1680
23
70.77
30
0.2462
29
88.07
26
0.2366
1
27
74.94
18
0.0163
28
82.61
23
0.0358
28
82.88
22
0.0378
2
22
68.30
17
0.0155
24
72.48
22
0.0351
27
74.56
21
0.0295
1
14
47.02
7
0.0024
10
40.74
5
0.0017
14
48.40
11
0.0052
2
19
62.26
7
0.0024
18
65.07
8
0.0019
26
69.87
9
0.0039
1
Table 20: Tabulation of median identification times for Class C. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an
identifier used to differentiate between the two submissions each participant could make. The Time column shows the time used to perform a search
over an enrollment set of 5 000 000. Time values are median times reported in seconds, but were originally recorded to microsecond precision. The FNIR
column shows FNIR for each submission at FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. The
number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
9
61
Accuracy Computational Resources Tradeoff
This section discusses the computational resources used by each submission, mainly looking for trends in accuracy of a
submission versus the load it created on the compute nodes. Statistics include how large the stored/finalized templates
are on disk versus in memory and how much time it took to create feature templates. Detailed tables in Appendix F show
enrollment set sizes, in Appendix G show search template sizes, and in Appendix H show template creation times. All
these tables were used to create the scatter plots used in this section. Appendix K plots relative comparisons of FNIR,
RAM usage, template creation times, and search times.
Addtionally, all the numbers in Tables 60 through 65 in Appendix F are based on the maximum enrollment set sizes for
each class shown in Table 4. The RAM values reported are the best estimate based on the information recorded. It is
possible that a submission used more or less RAM depending on the internal operations of the submitted software. See
the Lessons Learned for Large-Scale Testing section for more details.
9.1
Storage and Memory
It is important to note that the Actual RAM used (Appendix F) is the sum of the resident enrollment set sizes of identification stage one processes after returning from the identification stage one initialization method. More information on this
can be found in the FpVTE API [15].
Every attempt was made to run each submission on the minimum number of compute nodes needed to successfully
complete the evaluation. This generally meant multiple passes of running enrollment set finalization and redoing timing
validation tests to determine the minimum number of compute nodes needed. If too few compute nodes were used, the
submission would crash and not work properly.
When looking at the results, there are submissions like those from participant Q that had large finalized enrollment set,
but used a lot less Actual RAM during stage one identification. In fact, Q always ran on a single compute node despite
the Finalized storage size. Participant L, on the other hand, clearly compressed templates, so they used more Actual RAM
than Finalized storage. This behavior was pre-reported to NIST, which made it easier to plan ahead when testing the
submission.
9.1.1
Class A
Scatter plots comparing FNIR and computational resources used by index finger identification submissions are shown in
Figures 46 through 51.
Figures 52 through 53 shows a comparison of the templates (right and left index) as stored on disk with the actual size
used in RAM. For the majority of participants, the numbers were very similar but there were exceptions such as T, P, G,
L, and Q.
Some observations for Class A computational resources include:
. The most accurate submissions were Q, V, I, D and L2.
. The most accurate submissions used the same or less RAM as other submissions with I2 being an exception.
. It appears that high accuracy can be achieved without using a large amount of storage.
. Participant T’s submissions consistently used the least computational resources but with the least accuracy.
. Of the most accurate submissions, participant V used the least amount of storage space.
. Participant K consumed the most RAM and disk space, significantly higher than all other participants.
. Participant Q used the least amount of RAM and achieved the highest accuracy of the most accurate submissions.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
62
FPVTE – F INGERPRINT M ATCHING
Left Index
0
1
2
3
4
5
0.50
FNIR @ FPIR =10−3
●
T1
M1
●
●
M2
0.20
H1 ●● H2
P1
●
C1 ● C2
●
P2
G1
●
● G2
F1 ● F2
0.10
O1
J1 ●●●● O2
S2
● T2
●
J2
●
S1
0.05
●
●
U1
U2
K1
●
K2
E1 E2
●
L1
●
●
●
V1
● V2
Q2 ●●
Q1
0.02
I1
●
L2
●
I2
D2
D1
●
0.01
1
2
RAM Used for Enrollment Set (gigabytes)
3
Figure 46: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 100 000 subjects and RAM used for enrollment set for Class A — Left
Index. The color of the data point is used to show the On Disk Finalized Enrollment Size. The color scale for the On Disk Finalized Enrollment Size is at the
top of the plot and the units are in gigabytes. The data for the scatter plot comes from Table 60 in Appendix F. RAM Used for Enrollment Set is from the
RAM/Actual column and On Disk Finalized Enrollment is from the On Disk/Finalized column.
Right Index
0
1
2
3
4
5
0.50
FNIR @ FPIR =10−3
M1
●
● M2
0.20 ● T1
H2
P1
C1 ● H1
●
● P2
C2●
G1
0.10 F1 ●●
● G2
O1
F2 ●
J1 ●●● O2
● T2
J2
S2
●
0.05
●
U2
●
U1
K2
●
K1
E1 ● E2
●
L1
●
S1
●
0.02
0.01
V2
Q1 ●● ● V1
Q2
I1
●
● D2
D1
L2
●
I2
1
2
RAM Used for Enrollment Set (gigabytes)
3
Figure 47: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 100 000 subjects and RAM used for enrollment set for Class A — Right
Index. The color of the data point is used to show the On Disk Finalized Enrollment Size. The color scale for the On Disk Finalized Enrollment Size is at the
top of the plot and the units are in gigabytes. The data for the scatter plot comes from Table 61 in Appendix F. RAM Used for Enrollment Set is from the
RAM/Actual column and On Disk Finalized Enrollment is from the On Disk/Finalized column.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
63
Left Index
0
2000
4000
6000
8000
10000
0.50
●
T1
FNIR @ FPIR =10−3
M1 ●● M2
0.20
P1
●
●
P2
0.10
H1 ●● H2
C1 ● C2
F1 ●● F2
O1
J1 ●● ●● O2
J2
●
L1
0.05
U2
●
●
G2
U1 ●
G1
K1 ● K2
●
E1 ● E2
S2 ●
●
S1
●
T2
●
L2
0.02
I1
●
V1 ● V2
●
D1 D2
Q2
●
●
Q1
0.01
2000
●
I2
4000
6000
8000
Search Template Size Per Subject In RAM (bytes)
10000
Figure 48: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 100 000 subjects and search template size in RAM for Class A — Left
Index. The color of the data point is used to show Search Template Size On Disk. The color scale for Search Template Size On Disk is at the top of the plot
and the units are in bytes. On Disk comes from table Table 66 in Appendix G.
Right Index
0
2000
4000
6000
8000
10000
0.50
FNIR @ FPIR =10−3
M1 ●● M2
0.20
0.10
●
T1
P1
●
●
P2
0.05
C1 ● C2
F1 ●● F2
●
H2
● H1
O1
●
J1 ●● ● O2
J2
L1
U1
●
G2
U2 ●
G1
K2
K1 ●●
E2
E1
S2 ●
●
S1
●
T2
●
0.02
0.01
V2
●
D2
V1
●
D1
Q2
●
●
Q1
2000
L2
●
I1
4000
6000
8000
Search Template Size Per Subject In RAM (bytes)
I2 ●
10000
Figure 49: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 100 000 subjects and search template size in RAM for Class A — Right
Index. The color of the data point is used to show Search Template Size On Disk. The color scale for Search Template Size On Disk is at the top of the plot
and the units are in bytes. On Disk comes from table Table 66 in Appendix G.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
64
FPVTE – F INGERPRINT M ATCHING
Left and Right Index
0
10
20
30
40
50
60
70
0.100
H1 ● H2
FNIR @ FPIR =10−3
0.050
G1
●
F2
●
●
● T2 F1 ● C2
C1
O1
●
S1
O2
J1 ● J2
S2
●
0.020
P1
●
G2
●
●
●
K1
●
●
U2
P2
●
U1
●
K2
E1
●
●
E2
●
●
L1
0.010
●
L2
●
I1
0.005
Q1
●
Q2
0.002
V1
●
●
V2
0.001
D1
●
D2
●
I2
20
40
RAM Used for Enrollment Set (gigabytes)
60
Figure 50: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 1 600 000 subjects and RAM used for enrollment set for Class A — Left
and Right Index. The color of the data point is used to show the On Disk Finalized Enrollment Set size. The color scale for the On Disk Finalized Enrollment
Set is at the top of the plot and the units are in gigabytes. The data for the enrollment set size comes from Table 62 in Appendix F. RAM used for Enrollment
Set is from the RAM/Actual column and On Disk Finalized Enrollment Set is from the On Disk/Finalized column.
Left and Right Index
0
5000
10000
15000
20000
0.100
H1 ● H2
0.050
P1
●
FNIR @ FPIR =10−3
●
P2
F2
●
F1 ●
G1
●
U2
K1 ● ●● U1
●
S1 ●● K2 G2
E1
S2 ● ●● E2
● C2
C1
O1
●
●
0.020
●
L1
O2
J1 ● J2
●
T2
0.010
●
L2
0.005
0.002
0.001
Q1
●
Q2
D1
●
D2
●
I1
V1
●
●
V2
5000
10000
15000
Search Template Size Per Subject In RAM (bytes)
I2 ●
20000
Figure 51: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 1 600 000 subjects and search template size in RAM. The color of the
data point is used to show Search Template Size On Disk. The color scale for Search Template Size On Disk is at the top of the plot and the units are in bytes.
On Disk comes from Table 66 in Appendix G.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
65
Left and Right Index
L1
32 GB
Actual RAM Consumption
●
P
●
●
E
H
J
●
●●
I2
●
O
G
C
●
●
L2
●
S
●
●
I1
●
●
Q
●
V
●
●
●
U1
U2
K
D1
D2
●●
●
M
F
1 GB
32 MB
T
●
1 GB
4 GB
16 GB
64 GB
Finalized Directory Size
Figure 52: Comparison of enrollment size in RAM and on disk. The x- and y-axes use log scales. The data for enrollment size comes from Table 62 in
Appendix F. Actual RAM Consumption is from the RAM/Actual column and Finalized Directory Size is from the On Disk/Finalized column.
Left and Right Index
●
20 kB
I2
15 kB
RAM Size
I1
T
●
G
●
S E
10 kB
●●
U
●
●●
●
O
D
●
●
●
5 kB
C
L1
●
●
V
L2
K
H
J
●
●
●
P
M
●
F
Q
●
●
0B
0B
5 kB
10 kB
15 kB
20 kB
Disk Size
Figure 53: Comparison of search template size in RAM and on disk. On Disk comes from Table 66 in Appendix G.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
66
FPVTE – F INGERPRINT M ATCHING
9.1.2
Class B
Scatter plots comparing FNIR and computational resources used by IDFlat identification submissions are shown in Figures 54 through 55.
Figures 56 through 57 shows a comparison of the templates as stored on disk with the actual size used in RAM. Like class
A results, this plot highlights submissions where on disk and in RAM usage differed such as E, G, L, and Q.
Some observations for Class B computational resources include:
. The lowest RAM usage is also one of the top performers (Q).
. Other top performers (participants D and I) do not have the largest RAM usage.
. Participant I cut RAM usage in half with minimal drop in accuracy.
. Like Class A, high accuracy can be achieved while keeping RAM usage relatively low.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
IDFlat
●
●
C1
C2
0
67
100
200
300
400
500
600
0.2000
FNIR @ FPIR =10−3
0.1000
M1 ●●
M2
0.0500
F2
● F1
H1 ● H2
0.0200
0.0100
J1
●
● O1
O2 ●● ● L1
● V1
J2
V2 ● ●
D1
0.0050
E1 ●
0.0020
Q2
●
0.0010 Q1
●
I1
U2
●
S2
●
● S1
G1
●
●
U1
G2
●
●
L2
●
E2
●
D2
●
I2
0.0005
100
200
300
400
RAM Used for Enrollment Set (gigabytes)
500
Figure 54: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 3 000 000 subjects and RAM used for enrollment set for Class B —
Identification Flats. The color of the data point is used to show On Disk Finalized Enrollment Set size. The color scale for the on disk finalized enrollment
size is at the top of the plot and the units are in gigabytes. The data for the enrollment set size comes from Table 63 in Appendix F. RAM Used for
Enrollment Set is from the RAM/Actual column and On Disk Finalized Enrollment Set is from the On Disk/Finalized column.
0
10000
C1 ●●
IDFlat
C2
20000
30000
40000
50000
60000
70000
80000
0.2000
FNIR @ FPIR =10−3
0.1000
F1
M1 ●●● F2
M2
0.0500
H2
●
H1
0.0200
S2
●
S1 ●
0.0100
0.0050
●
E1
0.0020
0.0010
L2
●
●
L1
V1
●
●
V2
Q2
●
Q1
●
I1
D1
●
J1
●
O1 ●●
● O2
J2
●
U2
●
U1 G1
●
●
G2
●
E2
●
D2
●
I2
0.0005
20000
40000
60000
Search Template Size Per Subject In RAM (bytes)
80000
Figure 55: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 3 000 000 subjects and search template size in RAM for Class B —
Identification Flats. The color of the data point is used to show Search Template Size On Disk. The color scale for Search Template Size On Disk is at the top
of the plot and the units are in bytes. On Disk comes from Table 67in Appendix G.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
68
FPVTE – F INGERPRINT M ATCHING
Identification Flats
●
512 GB
●
U1
U2
●
L2
Actual RAM Consumption
256 GB
●
●
G
128 GB
E2
S
●
H
L1
●
E1
64 GB
●
●
V
I2
●
C
32 GB
●
●
F D
M
●
●
J
●
●
●
O
I1
16 GB
Q
8 GB
●
32 GB
64 GB
128 GB
256 GB
512 GB
Finalized Directory Size
Figure 56: Comparison of enrollment size in RAM and on disk for Class B — Identification Flats. The x- and y-axes use log scales. The data for
enrollment size comes from Table 63 in Appendix F. Actual RAM Consumption is from the RAM/Actual column and Finalized Directory Size is from the
On Disk/Finalized column.
Identification Flats
●
●
60 kB
●
G
●
S
●
E2
U2
U1
RAM Size
45 kB
●
O
30 kB
I1
H
●
M
15 kB
●
E1
●
●
●
●
●●
●
J
I2
D
V
●
F
C
L
Q
●
19 kB
38 kB
57 kB
76 kB
Disk Size
Figure 57: Comparison of search template size in RAM and on disk for Class B — Identification Flats. On Disk comes from Table 67 in Appendix G.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
9.1.3
69
Class C
Scatter plots comparing FNIR and computational resources used by plain and rolled impression submissions are shown
in Figures 58 through 61.
Figures 62 through 65 show a comparison of the templates as stored on disk with the actual size used in RAM. Like class
A and B results, these plot highlight submissions where on disk and in RAM usage differed for several submissions.
Some observations for Class C computational resources include:
. Like classes A and B, the top performers do not use the largest amount of RAM.
. High accuracy can be achieved with relatively low RAM usage.
. Ten-finger rolled data used more RAM than ten-finger plain data, but is not more accurate.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
70
FPVTE – F INGERPRINT M ATCHING
●
0.2000
200
400
500
600
700
800
900
M1
●
●
M2
F1 ●● C2
F2
0.0500
G1
●
S1 ● ●
G2
H1 ● H2
0.0200
0.0100
●
U1
●
●
U2
●
L1
●
E1
L2
J1
●
0.0050
Q2
●
● Q1
I1
●
●
●
●
●
E2
O2
● J2
●
O1
V1
V2 ●
0.0020
0.0010
300
S2 ●
0.1000
FNIR @ FPIR =10−3
Plain
C1
100
0
D1
D2
I2
0.0005
200
400
600
RAM Used for Enrollment Set (gigabytes)
800
Figure 58: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 5 000 000 subjects and RAM used for enrollment set for Class C —
Ten-Finger Rolled-to-Rolled. The color of the data point is used to show the On Disk Finalized Enrollment Set size. The color scale for the On Disk Finalized
Enrollment Size is at the top of the plot and the units are in gigabytes. The data for the scatter plot comes from Table 64 in Appendix F. RAM used for
Enrollment Set is from the RAM/Actual column and On Disk Finalized Enrollment Set is from the On Disk/Finalized column.
Rolled
0
100
200
300
400
500
600
700
800
900
●
S2
0.2000
FNIR @ FPIR =10−3
0.1000
M1
●
● M2
F1 ● F2
0.0500
●
S1
G1
●
U2
●
U1
●
G2
0.0200
H1 ● H2
E1
●
C1 ●●
C2
0.0100
L1
●
●
0.0050
0.0020 Q1
●
●
0.0010
0.0005
Q2
L2
J1
●
O1
O2 ●● J2
E2 ●
V2
●
D2
●
●
D1
● I2
●
I1
●
V1
200
400
600
RAM Used for Enrollment Set (gigabytes)
800
Figure 59: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 5 000 000 subjects and RAM used for enrollment set for Class C —
Ten-Finger Rolled-to-Rolled. The color of the data point is used to show the On Disk Finalized Enrollment Set size. The color scale for the On Disk Finalized
Enrollment Size is at the top of the plot and the units are in gigabytes. The data for the scatter plot comes from Table 65 in Appendix F. RAM used for
Enrollment Set is from the RAM/Actual column and On Disk Finalized Enrollment Set is from the On Disk/Finalized column.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Plain
C1
20000
●
0
40000
60000
100000
120000
40000
60000
80000
100000
Search Template Size Per Subject In RAM (bytes)
120000
0.2000
80000
140000
S2 ●
M1
●
M2 ●● F1●
F2 C2
0.1000
FNIR @ FPIR =10−3
71
0.0500
G1
●
S1 ● G2 ●
U1
●
●
U2
H2
H1
●
0.0200
L1
●
●
E1 ● L2
0.0100
J1
●
0.0050
V1
●
V2
0.0020
Q2
●
●
0.0010
Q1
0.0005
J2 ●● O2
O1
D1 ●
● D2
●
I1
●
E2
●
I2
20000
Figure 60: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 5 000 000 subjects and search template size in RAM for Class C —
Ten-Finger Plain-to-Plain. The color of the data point is used to show Search Template Size On Disk. The color scale for Search Template Size On Disk is at
the top of the plot and the units are in bytes. On Disk comes from Table 68 in Appendix G.
Rolled
0
20000
40000
60000
80000
0.1000
M1
M2 ●●
F1 ● F2
0.0500
0.0200
S1 ●
U2
●
U1
L1
●
E1 ●
●
L2
●
●
G1
G2
C1
●
●
C2
J1
●
0.0050
●
O2 O1
J2
●
●
0.0005
V2
●
Q1
●
●
0.0010
140000
H1 ● H2
0.0100
0.0020
120000
S2 ●
0.2000
FNIR @ FPIR =10−3
100000
Q2
●
●
●
I1 I2
20000
V1
D1 ●
●
E2
D2
40000
60000
80000
100000
Search Template Size Per Subject In RAM (bytes)
120000
Figure 61: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 5 000 000 subjects and search template size in RAM for Class C —
Ten-Finger Rolled-to-Rolled. The color of the data point is used to show Search Template Size On Disk. The color scale for Search Template Size On Disk is
at the top of the plot and the units are in bytes. On Disk comes from Table 68 in Appendix G.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
72
FPVTE – F INGERPRINT M ATCHING
Ten−Finger Plain Impression
●
E2
U
●
Actual RAM Consumption
512 GB
●
256 GB
S1
●
L2 G
●
128 GB
H
●
●
L1
●
●
O
●
●
E1
C
●●
●
V
●
F
64 GB
J
D
I1
I2
M
●
S2
32 GB
Q
16 GB
●
64 GB
128 GB
256 GB
512 GB
1 TB
Finalized Directory Size
Figure 62: Comparison of enrollment size in RAM and on disk for Class C — Plain Impression. The x- and y-axes use log scales. The data for enrollment
size comes from Table 64 in Appendix F. Actual RAM Consumption is from the RAM/Actual column and Finalized Directory Size is from the On
Disk/Finalized column.
Ten−Finger Plain Impression
●
G
64 kB
U
●
●
E2
●
S
RAM Size
48 kB
D2
32 kB
H
I2
I1
C
●
16 kB
E1
●
●
●
O
●●
●
●
J
D1
●
V
●
L F
●
M
Q
●
20 kB
40 kB
60 kB
80 kB
Disk Size
Figure 63: Comparison of search template size in RAM and on disk for Class C — Plain Impression. On Disk comes from Table 68 in Appendix G.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
73
Ten−Finger Rolled Impression
1 TB
●
E2
Actual RAM Consumption
512 GB
L2
●
●
128 GB
L1
O
●
H
E1
●
●
F
M
I1
C
●
D
●
●
●
●
V
G
●
●
S1
J
●
256 GB
U
●
I2
●
S2
64 GB
32 GB
Q
●
128 GB
256 GB
512 GB
1 TB
Finalized Directory Size
Figure 64: Comparison of enrollment size in RAM and on disk for Class C — Rolled Impression. The x- and y-axes use log scales. The data for
enrollment size comes from Table 65 in Appendix F. Actual RAM Consumption is from the RAM/Actual column and Finalized Directory Size is from the On
Disk/Finalized column.
Ten−Finger Rolled Impression
●
108 kB
E2
81 kB
S
●
RAM Size
D2
J
●
G
O
54 kB
●
●
C
I1
27 kB
H
E1
●
V
U
D1
●
●
F
●
L
●
●
●
●
●
I2
M
Q
●
33 kB
66 kB
99 kB
132 kB
Disk Size
Figure 65: Comparison of search template size in RAM and on disk for Class C — Rolled Impression. On Disk comes from Table 68 in Appendix G.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
74
FPVTE – F INGERPRINT M ATCHING
9.2
Processing Time
This section shows the time required to enroll fingerprints images also referred to as template creation time. The template
creation times were recorded by enrolling a common sample of the datasets on common hardware, with 100 processes
running in parallel across 10 compute nodes (10 processes per compute node). Any segmentation time for slap captures
was included in the template creation times.
The detailed tables (Tables 72 and 80) used to make the scatter plots (Figures 66 through 71) in this section are included in
Appendix H. The Enrollment columns give some idea as to the required system capacity needed to enroll a large dataset in
a reasonable time frame. As an example, if a submission takes the full time allowed per image (3 seconds) and 16 compute
nodes are used for the enrollment process, it will take approximately 16 days to process all the enrollment sets for all three
classes.
In an operational sense, enrollment only occurs a single time for the entire dataset, and then on an as-needed basis when
new subjects are added to the dataset. The Search columns are different, as they show the time needed to create a template
every time a new search is performed. This time would be factored in as part of the overall search time process.
Some observations from tables and plots include:
. The most accurate submissions did not have the fastest enrollment times. In fact, the best performers tend to have
longer enrollment times.
. Enrollment time appears to be proportional to finger type and impression. For example, single-finger captures are
faster than ten-finger plain impressions, which are faster than ten-finger rolled impressions.
. Segmentation does not appear to significantly increase enrollment times, as noted by comparing ten-finger plain
impressions to ten-finger rolled impressions.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
75
Left Index
0.0
0.5
1.0
1.5
2.0
2.5
3.0
0.50
●
FNIR @ FPIR =10−3
T1
0.20
U1
●
●
U2
0.10
●
0.05
L1
M1
●
●
M2
H2
H1 ●● P1
● C2
P2 ●●C1
G1 ●
G2
O1
● J1
O2 ●J2 ●●●●● E1
E2
T2
S2
●
K1
●
K2
F1
●
●
F2
●
S1
●
L2
0.02
V1
●
V2
0.01
0.5
●
●
Q2
●
●
Q1
I1
I2
D1
●
D2
1.0
1.5
Enrollment Template Creation Time (seconds)
2.0
Figure 66: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 100 000 subjects and enrollment template creation time for Class A —
Left Index. The color of the data point is used to show the search template creation time. The color scale for search template creation time is at the top of
the plot and the units are in seconds. The template creation time data is from Table 72 in Appendix H.
Right Index
0.0
0.5
1.0
1.5
2.0
2.5
3.0
FNIR @ FPIR =10−3
0.50
M1
●
●
M2
0.20
●
0.10
0.05
●
L1
T1
H2
● C2
H1P1
●
● ● C1
U1
●
P2
● G2
U2
G1
O1 ● J1
●
O2 ●J2 ● ● E1
E2
●
T2
S2
●
K1
●
K2
F1
●
●
F2
●
S1
●
L2
0.02
0.01
V1
●
V2
Q2
●
●
Q1
0.5
●
I1
●
I2
D1 ● ● D2
1.0
1.5
Enrollment Template Creation Time (seconds)
2.0
Figure 67: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 100 000 subjects and enrollment template creation time for Class A —
Right Index. The color of the data point is used to show the search template creation time. The color scale for search template creation time is at the top
of the plot and the units are in seconds. The template creation time data is from Table 73 in Appendix H.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
76
FPVTE – F INGERPRINT M ATCHING
Left and Right Index
0
1
2
3
4
5
0.100
FNIR @ FPIR =10−3
0.050
0.020
H1 ● H2
● G1
C2
U2
●
●
●
C1
T2 ●P1 ● P2
●
●
U1
G2
O1
●
O2 ● ●● E1
E2
●
J1 ● J2
L1
K1
●
S1
●
F2
●
●
F1
●
K2
●
S2
0.010
●
L2
●
I1
0.005
V1
●
Q1
●
Q2
●
V2
0.002
0.001
1
D2 ●● D1
2
3
Enrollment Template Creation Time (seconds)
●
I2
4
Figure 68: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 1 600 000 subjects and enrollment template creation time for Class A —
Left and Right Index. The color of the data point is used to show the search template creation time. The color scale for search template creation time is
at the top of the plot and the units are in seconds. The template creation time data is from Table 74 in Appendix H.
IDFlat
●
●
C1
5
C2
0
10
15
20
0.2000
FNIR @ FPIR =10−3
0.1000
F1
F2●●●M1
M2
0.0500
H2
●
H1
0.0200
U2
●
0.0100
0.0050
●
L2
0.0020
S2
●
●
●
J1
●
● O1
●
●
O2
● J2
L1
● V1
●
V2
S1
U1
● G1
E1 ● ● G2
E2
●
●
D1
0.0010
Q1
●
Q2
●
D2
●
I1
●
I2
0.0005
5
10
15
Enrollment Template Creation Time (seconds)
Figure 69: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 3 000 000 subjects and enrollment template creation time for Class B —
Identification Flats. The color of the data point is used to show the search template creation time. The color scale for search template creation time is at
the top of the plot and the units are in seconds. The template creation time data is from Table 78 in Appendix H.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Plain
●
0
5 C1
10
0.2000
15
20
25
30
25
30
●
S2
M1
●
●
F1 ●M2
F2
0.1000
●
FNIR @ FPIR =10−3
77
0.0500
C2
G1
●
0.0200
H1 ● ● G2
H2
U1 ●● U2
L1
●
●
L2
0.0100
0.0050
●
S1
●
E1
●
J1 ●
E2
O2
J2 ●●●O1
V2
V1
0.0020
●
D1
0.0010
Q2
●
●
0.0005
5
I1
●
●
Q1
●
D2
I2
10
15
20
Enrollment Template Creation Time (seconds)
Figure 70: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 5 000 000 subjects and enrollment template creation time for Class C –
Ten-Finger Plain-to-Plain. The color of the data point is used to show the search template creation time. The color scale for search template creation time
is at the top of the plot and the units are in seconds. The template creation time data is from Table 79 in Appendix H.
Rolled
0
5
10
15
20
25
30
●
S2
0.2000
0.1000
M1
●
● M2
●
F1 F2
●
FNIR @ FPIR =10−3
S1
0.0500
G1
●
U2
●
U1
●
G2
H1 ● H2
0.0200
0.0100
L1
●
●
E1
●
L2
J1
●
O1
●
●
O2 J2
0.0050
0.0020
●
C2
●
E2
●
V1 ●
V2
Q1
●
●
Q2
0.0010
0.0005
C1
●
5
●
D1
D2 ●
●
I2
●
I1
10
15
20
Enrollment Template Creation Time (seconds)
25
30
Figure 71: Scatter plot of FNIR @ FPIR = 10−3 searching 30 000 subjects against 5 000 000 subjects and enrollment template creation time for Class C –
Ten-Finger Rolled-to-Rolled. The color of the data point is used to show the search template creation time. The color scale for search template creation
time is at the top of the plot and the units are in seconds. The template creation time data is from Table 80 in Appendix H.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
78
10
FPVTE – F INGERPRINT M ATCHING
Ranked Results
This section combines tables from Sections 7 and 9 into a single table. The resulting tables are rank-sorted based on FNIR
values.
There is one table from each class of participation included in the main body of this report. The full set of tables for all
classes and search set scenarios are included in Appendix I.
These tables are useful because they combine all the high-level information in a single table. They are rank-sorted on FNIR,
as accuracy is generally considered the most important goal for an identification algorithm to achieve. The reader can then
look across and see how a participant ranked in other areas such as search time (Identification), search template creation
time (Search Enrollment), and memory usage (RAM). As stated in previous sections, the most accurate submissions are
not the fastest. In all three classes, there is a two to three times increase in the error rate when comparing the most accurate
submission with one of the top three fastest in search speed.
Appendix K plots relative comparisons of FNIR, RAM usage, template creation times, and search times.
Appendix L is one attempt to take the tables in this section and apply some relative weight or importance to each column.
The tables in Appendix L use these weights to produce a score for each submission and then sort the results based on
those scores. Refer to Appendix L for more details.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
≥ 20 seconds
< 20 seconds
Participant
Identification
Mean
Median
FNIR
79
Search Enrollment
Mean
Median
RAM
Letter
Sub. #
V
1
1
0.0034
5
9.50
5
9.29
4
0.68
3
0.64
5
11.77
I
1
2
0.0058
9
18.92
8
17.87
9
2.96
9
2.93
6
15.83
J
1
3
0.0143
6
14.00
6
13.64
3
0.66
4
0.65
3
8.30
L
1
4
0.0146
1
2.20
2
2.19
1
0.11
1
0.11
8
18.76
O
1
5
0.0229
7
15.34
7
14.46
2
0.55
2
0.53
4
9.05
K
1
6
0.0360
8
18.32
9
18.01
8
2.11
8
2.09
9
61.81
C
1
7
0.0368
2
2.39
1
2.08
5
0.80
5
0.78
2
4.87
C
2
8
0.0374
4
6.34
4
6.35
5
0.80
5
0.78
1
4.87
G
1
9
0.0515
3
6.27
3
5.22
7
1.13
7
1.09
7
16.38
Q
1
1
0.0027
21
213.08
21
212.69
16
1.13
15
1.08
10
9.14
Q
2
1
0.0027
19
163.65
19
161.02
16
1.13
15
1.08
9
9.14
V
2
3
0.0028
18
133.45
18
127.65
10
0.68
9
0.64
13
11.77
I
2
4
0.0030
25
385.14
25
338.88
27
4.37
27
4.30
21
30.83
D
2
4
0.0030
23
234.52
23
237.43
26
3.12
26
2.84
17
18.58
D
1
4
0.0030
16
73.01
15
70.99
25
3.10
25
2.84
17
18.58
L
2
7
0.0072
3
23.54
3
23.42
3
0.33
3
0.31
26
53.98
J
2
8
0.0143
7
36.19
7
33.35
9
0.66
10
0.65
7
8.30
S
2
9
0.0195
26
429.02
26
495.50
18
1.77
18
1.75
14
14.82
E
2
10
0.0202
27
500.30
27
518.11
11
0.70
11
0.70
23
33.41
E
1
11
0.0207
2
22.27
1
16.35
11
0.70
11
0.70
22
33.40
O
2
12
0.0214
10
45.52
10
43.53
4
0.55
4
0.53
8
9.05
0.0281
1
14.82
S
1
13
20.11
2
23.00
18
1.77
18
1.75
14
K
2
14
0.0286
6
32.68
6
32.93
20
2.11
20
2.09
27
61.81
G
2
15
0.0311
22
227.27
22
221.16
15
1.13
17
1.09
16
16.38
P
2
16
0.0333
17
114.37
17
101.16
13
0.77
13
0.72
19
21.41
U
1
17
0.0336
11
47.60
11
45.51
1
0.30
1
0.30
25
44.63
U
2
18
0.0358
24
252.04
24
240.40
1
0.30
1
0.30
24
37.35
T
2
19
0.0366
9
38.91
9
37.23
5
0.65
7
0.62
1
0.01
P
1
20
0.0370
14
63.41
14
63.65
13
0.77
13
0.72
20
21.42
F
1
21
0.0386
13
59.30
13
60.66
21
2.23
21
2.15
5
4.86
F
2
22
0.0412
15
71.45
16
73.95
21
2.23
21
2.15
5
4.86
H
2
23
0.0684
12
53.77
12
52.25
7
0.66
5
0.60
11
9.36
H
1
24
0.0686
8
37.65
8
36.71
7
0.66
5
0.60
12
9.36
M
2
25
NA
20
183.20
20
171.24
23
2.25
23
2.16
4
3.76
M
1
26
NA
5
32.59
5
31.44
23
2.25
23
2.16
3
3.75
T
1
27
NA
4
26.96
4
25.68
5
0.65
7
0.62
1
0.01
Table 21: Tabulation of ranked results for Class A — Left and Right Index. Submissions were split into two groups. The first group includes submissions
that performed searches on average in less than 20 seconds, and the second includes those that took, on average, 20 seconds or longer. Letter refers to the
participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could
make. The FNIR column was computed at the score threshold that gave FPIR = 10−3 . NA indicates that the operations required to produce the value
could not be performed. The Identification column shows the time used to perform a search over an enrollment set of 1 600 000, as seen in Table 18. The
Search Enrollment column shows the time used to create a search template to be used for a query, as seen in Table 74. Identification and Search Enrollment
durations are reported in seconds, but were originally recorded to microsecond precision. RAM refers to the sum of the resident set sizes of the stage
one identification processes over all compute nodes after returning from the identification stage one initialization method, as seen in Table 62. RAM is
reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s column-wise ranking, with the
best performance shaded in green and the worst in pink. The table is sorted on the FNIR column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
80
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
Sub. #
Identification
Mean
Median
FNIR
Search Enrollment
Mean
Median
RAM
I
2
1
0.0009
20
60.01
16
43.86
30
19.36
30
19.36
21
108.71
Q
1
2
0.0012
14
48.85
14
42.92
20
8.89
20
8.92
1
7.54
Q
2
2
0.0012
24
71.67
24
66.01
20
8.89
20
8.92
2
7.54
I
1
2
0.0012
6
24.57
7
19.07
25
16.50
25
16.52
5
49.68
D
2
2
0.0012
19
54.37
18
46.70
24
11.90
24
11.86
13
79.43
D
1
6
0.0020
17
52.42
17
45.15
17
6.19
17
6.16
12
79.43
V
2
7
0.0024
15
49.72
19
49.30
3
3.35
7
3.36
9
63.53
E
2
7
0.0024
18
52.88
15
43.61
16
5.95
16
5.93
28
317.95
V
1
9
0.0027
10
35.51
10
34.96
3
3.35
7
3.36
8
63.53
L
1
10
0.0031
5
14.42
5
14.50
2
3.05
2
3.05
22
119.79
L
2
11
0.0033
8
28.56
9
28.59
1
0.88
1
0.88
27
177.48
J
2
11
0.0033
22
64.38
22
60.13
7
3.38
3
3.35
17
101.00
O
2
13
0.0035
21
63.87
21
60.02
5
3.38
5
3.36
20
101.00
G
2
14
0.0040
9
31.67
6
16.26
18
7.89
18
7.90
26
156.12
O
1
15
0.0041
11
37.04
11
35.64
5
3.38
5
3.36
19
101.00
E
1
16
0.0043
2
8.76
2
6.76
13
5.35
13
5.34
14
79.73
J
1
17
0.0049
7
26.31
8
25.38
7
3.38
3
3.35
18
101.00
G
1
18
0.0062
1
6.33
1
4.26
18
7.89
18
7.90
25
156.12
U
1
19
0.0099
30
88.83
28
86.60
15
5.90
15
5.82
29
440.68
S
1
20
0.0108
28
86.50
29
87.60
22
10.24
22
10.31
24
150.35
S
2
21
0.0136
29
88.70
30
88.46
22
10.24
22
10.31
23
150.35
U
2
22
0.0141
25
80.14
25
75.28
14
5.80
14
5.73
30
540.60
H
1
23
0.0203
26
82.43
26
82.66
9
4.37
9
4.37
15
86.46
H
2
24
0.0204
27
85.74
27
86.56
9
4.37
9
4.37
16
86.46
M
2
25
0.0515
23
70.45
23
65.91
26
18.36
28
18.11
6
57.53
M
1
26
0.0543
13
41.37
13
38.87
26
18.36
28
18.11
6
57.53
F
1
27
0.0591
12
39.12
12
37.56
28
18.36
26
18.07
10
77.02
F
2
27
0.0591
16
51.59
20
49.34
28
18.36
26
18.07
10
77.02
C
2
29
NA
4
10.28
4
10.21
11
5.16
11
5.13
4
46.49
C
1
30
NA
3
9.00
3
7.92
11
5.16
11
5.13
3
46.49
Table 22: Tabulation of ranked results for Class B — Identification Flats. Letter refers to the participant’s letter code found on the footer of this page. Sub.
# is an identifier used to differentiate between the two submissions each participant could make. The FNIR column was computed at the score threshold
that gave FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. The Identification column shows the
time used to perform a search over an enrollment set of 3 000 000, as seen in Table 19. The Search Enrollment column shows the time used to create a
search template to be used for a query, as seen in Table 78. Identification and Search Enrollment durations are reported in seconds, but were originally
recorded to microsecond precision. RAM refers to the sum of the resident set sizes of the stage one identification processes over all compute nodes after
returning from the identification stage one initialization method, as seen in Table 63. RAM is reported in gigabytes, where 1 GB is equal to 1 073 741 824
bytes. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink.
The table is sorted on the FNIR column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
Sub. #
Identification
Mean
Median
FNIR
81
Search Enrollment
Mean
Median
RAM
I
1
1
0.0013
24
79.04
15
54.56
25
20.21
25
20.23
4
113.80
Q
2
2
0.0014
27
83.35
25
74.22
17
12.95
17
13.06
2
20.22
I
2
2
0.0014
9
40.53
7
30.82
24
18.72
24
18.73
11
137.03
D
1
4
0.0015
17
65.97
17
58.92
23
17.44
21
17.39
10
132.40
Q
1
5
0.0017
26
83.25
26
74.40
17
12.95
17
13.06
1
20.22
V
1
5
0.0017
10
40.94
10
40.74
9
8.48
9
8.50
17
234.03
D
2
7
0.0018
30
86.39
27
74.97
30
30.43
30
30.47
9
132.37
V
2
8
0.0019
16
65.47
18
65.07
9
8.48
9
8.50
18
234.03
O
2
9
0.0033
19
72.55
21
69.30
6
6.80
6
6.71
23
303.13
J
2
9
0.0033
20
72.62
19
67.98
8
6.82
8
6.74
24
303.13
O
1
11
0.0034
15
59.01
14
54.42
6
6.80
6
6.71
25
303.13
E
2
12
0.0050
14
57.02
11
42.73
11
9.89
11
9.91
30
930.48
J
1
13
0.0051
8
36.09
9
33.02
5
6.78
5
6.69
22
303.13
L
2
14
0.0083
3
19.52
5
20.25
3
4.36
3
4.36
26
367.82
C
2
15
0.0085
5
25.91
6
21.97
13
10.79
13
10.69
15
183.36
C
1
16
0.0094
4
25.25
4
20.15
13
10.79
13
10.69
14
183.36
L
1
17
0.0097
7
31.42
8
31.68
3
4.36
3
4.36
21
280.49
E
1
18
0.0106
1
9.52
2
8.63
12
9.90
12
9.92
16
191.66
H
2
19
0.0199
29
84.26
29
84.50
15
12.08
15
12.17
12
144.02
H
1
20
0.0201
28
84.14
30
84.51
15
12.08
15
12.17
13
144.02
G
2
21
0.0333
6
30.61
3
19.46
19
15.55
19
15.50
19
261.29
U
2
22
0.0351
23
74.39
24
72.48
1
2.94
1
2.87
28
806.17
U
1
23
0.0358
25
82.76
28
82.61
1
2.94
1
2.87
28
806.17
G
1
24
0.0447
2
11.67
1
7.78
19
15.55
19
15.50
20
261.29
F
1
25
0.0536
13
55.63
16
55.79
28
21.07
28
20.92
7
130.29
F
2
25
0.0536
18
68.61
20
68.20
28
21.07
28
20.92
6
130.29
M
2
27
0.0716
12
48.71
13
48.80
26
21.02
26
20.89
5
130.29
M
1
28
0.0783
11
47.38
12
48.08
26
21.02
26
20.89
8
130.29
S
1
29
0.0860
22
74.01
22
69.71
21
17.43
22
17.57
27
382.88
S
2
30
0.2462
21
72.95
23
70.77
21
17.43
22
17.57
3
78.28
Table 23: Tabulation of ranked results for Class C — Ten-Finger Rolled-to-Rolled. Letter refers to the participant’s letter code found on the footer of this
page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. The FNIR column was computed at the
score threshold that gave FPIR = 10−3 . The Identification column shows the time used to perform a search over an enrollment set of 5 000 000, as seen
in Table 20. The Search Enrollment column shows the time used to create a search template to be used for a query, as seen in Table 80. Identification and
Search Enrollment durations are reported in seconds, but were originally recorded to microsecond precision. RAM refers to the sum of the resident set
sizes of the stage one identification processes over all compute nodes after returning from the identification stage one initialization method, as seen in
Table 65. RAM is reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s column-wise
ranking, with the best performance shaded in green and the worst in pink. The table is sorted on the FNIR column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
82
FPVTE – F INGERPRINT M ATCHING
11
How Many Fingers are Needed
It is already well known that using more fingers results in a lower FNIR [17]. This section combines the results from
Figures 12, 15, 18, and 21 into a single plot in Figure 72.
The reader is reminded that enrollment set sizes were 100 000 subjects for single index fingers, 1.6 million subjects for two
index fingers, 3 million subjects for IDFlats, and 5 million for ten-finger rolled and plain impressions. The search set size
was 30 000 that included 20 000 nonmate and 10 000 mate searches.
Additionally, Appendix J plots relative comparisons, by class, for each search set used in FpVTE.
Some observations regarding numbers of fingers include:
. More fingers were better and produced the most accurate results.
. More fingers took more time to enroll (Subsection 9.2)
. Ten-finger plain-to-plain impressions were as accurate as ten-finger rolled-to-rolled impressions with higher performing submissions.
. More fingers generally produce faster search times against very large enrollment sets (Section 8).
. Class B four-finger slap identification appeared to be less accurate than Class A two-finger identification. This needs
further investigation as to the cause. Two possibilities are slap segmentation errors or fingerprint image quality.
After manually inspecting some of the errors and considering the ten finger IDFlat and plain-to-plain results, it
would appear that image quality may have been the largest contributing factor.
●
●
0.200
●
●
●
0.100
FNIR @ FPIR =10−3
●
●
●
●
0.050
●
0.020
●
●
●
●
●
●
●
●
●
0.010
●
●
●
●
●
●
●
●
●
0.005
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0.002
0.001
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
I2 D2 I1 Q1 Q2 D1 E2 V2 V1 L1 J2 L2 O2 G2 O1 E1 J1 G1 U1 S1 S2 U2 H1 H2 M2 M1 F1 F2 C2 C1 K1 K2 P1 P2 T1 T2
Submission
●
Identification Flats
●
Left and Right Slap
●
Left Slap
●
Right Slap
●
Ten−Finger Plain−to−Rolled
●
Left and Right Index
●
Left Index
●
Right Index
●
Ten−Finger Plain−to−Plain
●
Ten−Finger Rolled−to−Rolled
Figure 72: Rank-sorted FNIR @ FPIR = 10−3 for All Classes. Submissions “1” and “2” from round 3.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
12
83
FpVTE 2003 Comparison
FpVTE 2003 [17] was composed of three separate tests, the Large-Scale Test (LST), the Medium-Scale Test (MST), and the
Small-Scale Test (SST). SST and MST tested matching accuracy using individual fingerprints, all of which were images
from right index fingers. This contrasts with LST, which evaluated matching accuracy using sets of fingerprint images,
where each set includes one to ten finger positions collected from an individual subject at one time.
LST used 64 000 fingerprint sets from 25 000 subjects. These fingerprint sets comprised multiple test sets with varying
combinations of one, two, four, eight, and ten fingers. MST used 10 000 right index fingers and SST used a subset of 1 000
right index fingers.
A significant difference between FpVTE 2012 and FpVTE 2003 testing procedures was that FpVTE 2003 required participants to match all subjects in the datasets against each other and return all 1-to-1 match scores. Therefore, while a direct
comparison of results from the two FpVTE evaluations is not possible, this section will look at some of the observations
from 2003 and note changes that have occurred in FpVTE 2012.
Looking at Figures 73 through 75 (focusing on the “Standard Partition” and “Average TAR”) , a notable observation is that
the accuracy gap in 2003 between the most accurate and least accurate systems was very significant. In current results,
there is still a measurable accuracy gap, but it doesn’t seem to be nearly as large.
In 2003, the accuracy results (as shown in Figure 76) indicated some difficulty measuring the accuracy difference when
using four-, eight-, and ten-finger datasets. While it was clear that more fingers produced higher accuracy, it was not clear
if ten fingers was significantly better than four fingers. The current FpVTE results used large enough datasets to allow a
more accurate measurement of four-, eight-, and ten-finger search sets. There was a noticeable improvement in matching
accuracy going from a four-finger search set to a ten-finger search set in FpVTE 2012.
Effects of fingerprint quality will be analyzed in a different FpVTE report to see if current technologies have improved
when using low quality fingerprint data.
Figure 73: FpVTE 2003 SST Results - Range of accuracy on single-finger flats (SST). These systems are sorted by accuracy
on the SST standard partition. Note that these results are reported at FAR = 10−3 [17].
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
84
FPVTE – F INGERPRINT M ATCHING
Figure 74: FpVTE 2003 MST Results - Range of accuracy across 7 MST partitions. These systems are sorted by accuracy on
the standard MST, which is simply the combination of the other partitions [17].
Figure 75: FpVTE 2003 LST Results - Range of accuracy over 27 operational LST partitions. The systems are sorted by their
average accuracy over the 27 partitions. Note that sorting by median accuracy would change the order for some systems
[17].
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
Figure 76: FpVTE 2003 LST Number of Finger Results - Effect of number of fingers and other variables in LST. The y scale is the log of False Reject Rate
(FRR), which is 1 − TAR. Note that the single-finger searches (red) are clearly separated from the two-finger searches (green), but the four-, eight-, and
ten-finger searches are intermingled. At the test sizes used, accuracy of four-, eight-, and ten-finger searches is difficult to differentiate. The lines off of
the top of the chart are for FRR = 0 (perfect results), which cannot be represented in log scale [17].
FPVTE – F INGERPRINT M ATCHING
85
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
86
13
FPVTE – F INGERPRINT M ATCHING
Lessons Learned for Large-Scale Testing
. Failure Feedback: One of the most difficult aspects of validation was providing useful feedback to participants when
failures occurred. The data used in FpVTE was sequestered operational data that could not be shared with participants. Some progress was made in FpVTE by allowing participants to write text-only logs, from their submission
executed at NIST, that could then be returned for analysis. NIST reviewed all logs to ensure the logs did not include
information related to the imagery or the NIST internal computing environment. Enhancements could be made to
the FpVTE API to allow the FpVTE test driver to toggle logging on and off, preventing participants from submitting
a separate logging build while maintaining the speed of not logging under normal use.
. Two-Stage Matching Data Transfer: The FpVTE API specifies a 4 GB RAM disk to allow submissions to write data
during the first stage of identification that could be referred to during the second stage. The intent was for all
processes running on a compute node to share the same 4 GB RAM disk. As NIST did not say how many processes
would be run in parallel, it created problems for submissions that wrote a large amount of data per process during
the first stage of identification. Without increasing the RAM disk size, NIST was forced to run fewer processes on
each compute node to avoid overfilling the RAM disk, which increased the evaluation time and wasted compute
node resources. For instance, a compute node that would typically run twenty stage one identification processes
might be limited to running just two or three. Future evaluations should better define the expected RAM disk usage,
a minimum number of concurrent processes, and other requirements needed to safely and effectively run multiple
searches in parallel.
. Shared Memory: A key feature of the FpVTE API was that a large enrollment set could be loaded in memory and
shared among multiple processes in parallel. An important aspect in allowing this shared memory useage was
that the memory must remain static after initialization (see Section 5 for details). This caused problems for some
participants and took several validation iterations to correct.
. Additional Computational Statistics: The FpVTE test driver recorded the resident enrollment set size of identification stage one processes after returning from the identification stage one initialization method. It was expected
that participants would use this method to load the entire enrollment set partition into RAM. While this was a
fairly good indicator of RAM requirements, some submissions allocated significantly more memory during the core
identification stage one method, which in many cases required re-partitioning the enrollment set with an additional
compute node. Should FpVTE be repeated, it would be more fair to record additional computational statistics, such
as peak RAM consumption over the execution time of the submission, since the RAM usage after initialization did
not completely represent the RAM resources required for some submissions to run.
. Timing Submissions: Keeping timing fair is a difficult task. The baseline of performing a timing test with only a
single FpVTE process running proved most successful at keeping timing fair for all participants. The timing test was
run against the full enrollment set with the assumption that using a smaller enrollment set would cause the search
times to decrease. In cases where any unusual results were noticed, the timing tests were repeated to verify that the
results remained consistent.
. Enrollment and Re-enrollment: Enrolling the full datasets (≈ 11.4 million total subjects across all three classes) was
not a trivial task. The original assumption was that this enrollment would only be performed for the first submission and not need to be repeated with later submissions. This was not the case and greatly increased the overall time
required to complete the evaluation. Any future evaluations of this magnitude should explore performing a “maximum size” template extraction up front, then allowing for adjustments during the finalization stage to only use the
minimum amount of information needed by the submissions. This could greatly reduce the need for re-enrollments.
Care would need to be taken when reporting the “size” of the enrollment templates for each submission.
. Enrollment Size — Disk vs. RAM: Another failed assumption was that reporting the size of the template at extraction would be a good indicator of how much RAM the enrollment set required (i.e., the Actual RAM and Reported
RAM columns from the tables Section 9 should be relatively close). For most submissions, this was true. Other
submissions either compressed the templates and required more memory than it appeared they would need (at least
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
87
one participant warned NIST that this would be the case) or they required less storage than the template sizes indicated they would need (again, at least one participant sent a warning about this issue). While the FpVTE API tried
to prevent this by asking for both memory usage in RAM and on disk, there was confusion among the participants
on what values, if any, to return. For example, many participants were confused on how to report memory usage of
slap images—the participant would segment the fingers and return a single template, but report back four separate
RAM usage values. Future evaluations will need to provide better guidance on this issue.
. Consolidations of Nonmate Searches: A large amount of unexpected time was spent performing consolidations on
the “back-end” of the searches. There proved to be a lot more consolidations to examine than originally expected. It
took two to three months to clean these up before meaningful results could be produced. A significant improvement
to this issue was the decision to flip nonmate search images, as discussed in Subsection 6.5.
. Operational Sequestered Data: Participants were able to learn things about their specific submissions even though
they may not have been one of the top performers. Some participants shared with NIST that they were grateful to test
on the large sample of sequestered operational data to which they might not otherwise have access. They may not
have had the best performance, but they were looking to learn about limitations with their submissions and make
improvements, which was one of stated goals of FpVTE. This should continue to help advance fingerprint matching
technologies and support the NIST mission of, “promoting U.S. innovation and industrial competitiveness”.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
88
FPVTE – F INGERPRINT M ATCHING
14
Way Forward
NIST plans to publish further research and analysis in addition to this initial results report, including:
. running larger search sets (300 000 nonmates and 50 000 mates) so that DET curves can show accuracy with FPIR
rates below 10−3 ;
. performing failure analysis in an attempt to determine if there are common failures among submissions and what
causes those failures. Some things to examine during failure analysis include image quality, segmentation errors,
gender differences, and consolidation errors;
. performing analysis of results based on NFIQ values for the datasets;
. looking at accuracy of subgroups of metadata, such as male versus female.
. performing further analysis and testing to determine possible causes for four-finger IDFlat slap images being less
accurate than two-index finger single-capture images;
If FpVTE were to be repeated, it might be useful to concentrate more on throughput versus accuracy. For instance, rather
than set a single maximum search time of 90 seconds, the evaluation could have several search time maximums in an effort
to see how different search times impact matching accuracy. It was clear during this evaluation that some participants have
finer-grained control over the speed in which searches were performed. A speed-vs-accuracy track/competition would
be useful.
14.1
Related Testing
14.1.1
Forensic Palmprint
As data becomes available, the protocols from this evaluation could be applied to perform an evaluation for latent palmprint matching.
14.1.2
Mobile Data
NIST has performed some testing with simulated mobile data, but future evaluations should look at using operational
mobile data in the search sets to see how it impacts performance of matching algorithms.
14.1.3
Cross-Comparison of Modalities
Additional testing will compare the performance of fingerprint, face, and iris matching algorithms, in which all use a
search set of 1.6 million subjects. While the datasets will be captured from different sources and subjects, this will be one
of the first steps in comparing different modalities on similar sample sizes.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
89
References
[1] C HAMBERS , J., C LEVELAND , W., K LEINER , B., AND T UKEY, P. Graphical Methods for Data Analysis. Wadsworth
Brooks/Cole, 1983. 198
[2] C RIMINAL J USTICE I NFORMATION S ERVICES D IVISION — F EDERAL B UREAU OF I NVESTIGATION. IAFIS-IC-0110
(V3.1) WSQ Gray-scale Fingerprint Image Compression Specification, 2010. 6
[3] G ROTHER , P., M ICHAELS , R., AND P HILLIPS , P. J. Face Recognition Vendor Test 2002 Performance Metrics. In Audioand Video-Based Biometric Person Authentication: 4th International Conference, AVBPA 2003, Guildford, UK, June 9–11, 2003
Proceedings (June 2003), J. Kittler and M. S. Nixon, Eds., vol. 2688, Springer Berlin Heidelberg, pp. 937–945. 1
[4] G ROTHER , P., Q UINN , G., M ATEY, J., N GAN , M., S ALAMON , W., F IUMARA , G., AND WATSON , C. Iris Exchange III.
NIST Interagency Report 7836 (2012). 18
[5] I MAGE G ROUP — NIST.
evaluations.cfm. 3
NIST Biometric Evaluations Website.
http://www.nist.gov/itl/iad/ig/biometric_
[6] I NTERNATIONAL C OMMITTEE FOR I NFORMATION T ECHNOLOGY S TANDARDS. American National Standard for Information Technology - Finger Minutiae Format for Data Interchange, ANSI/INCITS 378-2004, 2004. 3
[7] ISO/IEC JTC 1 SC 37. Information technology – Vocabulary – Part 37: Biometrics. http://standards.iso.org/
ittf/PubliclyAvailableStandards/c055194_ISOIEC_2382-37_2012.zip, 2012. 1
[8] K OMARINSKI , P. Automated Fingerprint Identification Systems (AFIS), 1st ed. Elsevier, 2004. ISBN 9780080475981. 19
[9] M ARTIN , A. F., D ODDINGTON , G. R., T ERRI K AMM , M. O., AND P RZYBOCKI , M. A. The DET curve in assessment
of detection task performance, 1997. 18
[10] O PEN MPI P ROJECT. Open MPI: Open Source High Performance Computing. http://www.open-mpi.org/, 2014.
[Online; accessed 22 October 2014]. 11
[11] S ALAMON , W., AND F IUMARA , G. Biometric Evaluation Framework. http://www.nist.gov/itl/iad/ig/framework.
cfm, 2014. [Online; accessed 28 August 2014]. 11
[12] TABASSI , E., AND W ILSON , C. L. A novel approach to fingerprint image quality. In Image Processing, 2005. ICIP 2005.
IEEE International Conference on (Sept 2005), vol. 2, pp. II–37–40. 6
[13] TABASSI , E., W ILSON , C. L., AND WATSON , C. I. Fingerprint Image Quality. NIST Interagency Report 7151 (2004). 6
[14] WATSON , C., G ARRIS , M. D., TABASSI , E., W ILSON , C. L., M C C ABE , R. M., J ANET, S., AND K O , K. User’s Guide to
Export Controlled Distribution of NIST Biometric Image Software (NBIS-EC). NIST Interagency Report 7391 (2007). 6
[15] WATSON , C., S ALAMON , W., AND F IUMARA , G. Fingerprint Vendor Technology Evaluation Implementer’s Guide. National Institute of Standards and Technology, http://nigos.nist.gov:8080/evaluations/fpvte2012/FPVTE2012_API_
Plan_full_060712.pdf, July 2012. xvi, 1, 61
[16] WAYMAN , J., J AIN , A., M ALTONI , D., AND M AI , D. Biometric Systems: Technology, Design and Performance
Evaluation, 2005. ISBN 1852335963. 19
[17] W ILSON , C., H ICKLIN , R. A., B ONE , M., K ORVES , H., G ROTHER , P., U LERY, B., M ICHEALS , R., Z OEPFL , M., O TTO ,
S., AND WATSON , C. Fingerprint Vendor Technology Evaluation 2003. NIST Interagency Report 7123 (2004). 82, 83,
84, 85
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
90
A
FPVTE – F INGERPRINT M ATCHING
Individual Participant FNIR Plots
This appendix contains a full size DET curve for every participant in FpVTE. The reader is reminded that enrollment set
sizes were 100 000 subjects for single index fingers, 1.6 million subjects for two index fingers, 3 million subjects for IDFlats,
and 5 million for ten-finger rolled and plain impressions.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.0005
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.5000
0.0005
0.0020
0.0050
0.0200
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
False Positive Identification Rate
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.1000
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
Figure 77: DET for Participant C — All fingers with maximum enrollment sets for each class.
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.5000
FPVTE – F INGERPRINT M ATCHING
91
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
False Negative Identification Rate
FPVTE – F INGERPRINT M ATCHING
92
False Negative Identification Rate
0.5000
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.0005
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.0005
0.0020
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.0050
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
0.0200
False Positive Identification Rate
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
0.1000
Figure 78: DET for Participant D — All fingers with maximum enrollment sets for each class.
0.5000
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.0005
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.5000
0.0005
0.0020
0.0050
0.0200
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
False Positive Identification Rate
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.1000
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
Figure 79: DET for Participant E — All fingers with maximum enrollment sets for each class.
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.5000
FPVTE – F INGERPRINT M ATCHING
93
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
False Negative Identification Rate
FPVTE – F INGERPRINT M ATCHING
94
False Negative Identification Rate
0.5000
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.0005
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.0005
0.0020
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.0050
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
0.0200
False Positive Identification Rate
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
0.1000
Figure 80: DET for Participant F — All fingers with maximum enrollment sets for each class.
0.5000
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.0005
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.5000
0.0005
0.0020
0.0050
0.0200
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
False Positive Identification Rate
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.1000
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
Figure 81: DET for Participant G — All fingers with maximum enrollment sets for each class.
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.5000
FPVTE – F INGERPRINT M ATCHING
95
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
False Negative Identification Rate
FPVTE – F INGERPRINT M ATCHING
96
False Negative Identification Rate
0.5000
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.0005
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.0005
0.0020
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.0050
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
0.0200
False Positive Identification Rate
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
0.1000
Figure 82: DET for Participant H — All fingers with maximum enrollment sets for each class.
0.5000
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.0005
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.5000
0.0005
0.0020
0.0050
0.0200
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
False Positive Identification Rate
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.1000
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
Figure 83: DET for Participant I — All fingers with maximum enrollment sets for each class.
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.5000
FPVTE – F INGERPRINT M ATCHING
97
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
False Negative Identification Rate
FPVTE – F INGERPRINT M ATCHING
98
False Negative Identification Rate
0.5000
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.0005
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.0005
0.0020
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.0050
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
0.0200
False Positive Identification Rate
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
0.1000
Figure 84: DET for Participant J — All fingers with maximum enrollment sets for each class.
0.5000
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.0005
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.5000
0.0005
0.0020
0.0050
0.0200
0.0500
Right Index (2)
Left and Right Index (2)
False Positive Identification Rate
0.0100
Left and Right Index (1)
Left Index (2)
0.1000
Figure 85: DET for Participant K — All fingers with maximum enrollment sets for each class.
0.0010
Left Index (1)
Right Index (1)
0.2000
0.5000
FPVTE – F INGERPRINT M ATCHING
99
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
False Negative Identification Rate
FPVTE – F INGERPRINT M ATCHING
100
False Negative Identification Rate
0.5000
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.0005
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.0005
0.0020
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.0050
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
0.0200
False Positive Identification Rate
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
0.1000
Figure 86: DET for Participant L — All fingers with maximum enrollment sets for each class.
0.5000
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.0005
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.5000
0.0005
0.0020
0.0050
0.0200
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
False Positive Identification Rate
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.1000
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
Figure 87: DET for Participant M — All fingers with maximum enrollment sets for each class.
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.5000
FPVTE – F INGERPRINT M ATCHING
101
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
False Negative Identification Rate
FPVTE – F INGERPRINT M ATCHING
102
False Negative Identification Rate
0.5000
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.0005
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.0005
0.0020
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.0050
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
0.0200
False Positive Identification Rate
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
0.1000
Figure 88: DET for Participant O — All fingers with maximum enrollment sets for each class.
0.5000
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.0005
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.5000
0.0005
0.0020
0.0050
0.0200
0.0500
Right Index (2)
Left and Right Index (2)
False Positive Identification Rate
0.0100
Left and Right Index (1)
Left Index (2)
0.1000
Figure 89: DET for Participant P — All fingers with maximum enrollment sets for each class.
0.0010
Left Index (1)
Right Index (1)
0.2000
0.5000
FPVTE – F INGERPRINT M ATCHING
103
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
False Negative Identification Rate
FPVTE – F INGERPRINT M ATCHING
104
False Negative Identification Rate
0.5000
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.0005
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.0005
0.0020
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.0050
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
0.0200
False Positive Identification Rate
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
0.1000
Figure 90: DET for Participant Q — All fingers with maximum enrollment sets for each class.
0.5000
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.0005
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.5000
0.0005
0.0020
0.0050
0.0200
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
False Positive Identification Rate
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.1000
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
Figure 91: DET for Participant S — All fingers with maximum enrollment sets for each class.
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.5000
FPVTE – F INGERPRINT M ATCHING
105
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
False Negative Identification Rate
FPVTE – F INGERPRINT M ATCHING
106
False Negative Identification Rate
0.5000
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.0005
0.0005
0.0010
Left Index (1)
Right Index (1)
0.0020
0.0100
Left and Right Index (1)
Left Index (2)
0.0050
0.0200
Right Index (2)
Left and Right Index (2)
0.0500
False Positive Identification Rate
0.1000
Figure 92: DET for Participant T — All fingers with maximum enrollment sets for each class.
0.2000
0.5000
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.0005
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.5000
0.0005
0.0020
0.0050
0.0200
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
False Positive Identification Rate
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.1000
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
Figure 93: DET for Participant U — All fingers with maximum enrollment sets for each class.
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.5000
FPVTE – F INGERPRINT M ATCHING
107
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
False Negative Identification Rate
FPVTE – F INGERPRINT M ATCHING
108
False Negative Identification Rate
0.5000
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.0005
0.0010
Left Index (1)
Right Index (1)
Left and Right Index (1)
Right Slap (1)
Left Slap (1)
0.0005
0.0020
0.0100
Left and Right Slap (1)
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
Ten−Finger Plain−to−Rolled (1)
0.0050
0.0500
Left Index (2)
Right Index (2)
Left and Right Index (2)
Right Slap (2)
Left Slap (2)
0.0200
False Positive Identification Rate
0.2000
Left and Right Slap (2)
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
Ten−Finger Plain−to−Rolled (2)
0.1000
Figure 94: DET for Participant V — All fingers with maximum enrollment sets for each class.
0.5000
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
B
109
Combined Class DETs and CMCs
This appendix contains DET and CMC curves for all classes and participants grouped together. There is one grouping for
the participant’s first submission and another for their second submission. The submissions are split for visibility only—
“first” and “second” submissions do not imply any sort of logical grouping. The reader is reminded that enrollment set
sizes were 100 000 subjects for single index fingers, 1.6 million subjects for two index fingers, 3 million subjects for IDFlats,
and 5 million for ten-finger rolled and plain impressions.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
110
FPVTE – F INGERPRINT M ATCHING
False Negative Identification Rate
0.5000
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.0005
0.5000
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.0005
P
I
C
Q
J
D
S
K
E
T
L
F
U
M
G
V
O
H
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
Left Index (1)
Right Index (1)
Left and Right Index (1)
0.0005
0.0010
0.0020
0.0005
0.0010
0.0020
0.0050
0.0100
0.0200
0.5000
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
0.0500
0.1000
0.2000
Right Slap (1)
Left Slap (1)
Left and Right Slap (1)
False Positive Identification Rate
0.0005
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.5000
Ten−Finger Plain−to−Rolled (1)
Figure 95: DETs from the first submission for all participants in all classes.
0.5000
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.0020
0.0010
0.0005
0.0200
0.0100
0.0050
0.2000
0.1000
0.0500
0.5000
0.0020
0.0010
0.0005
0.0200
0.0100
0.0050
P
I
0.0050
0.0100
0.0200
0.2000
0.1000
0.0500
C
S
K
E
0.0500
0.1000
0.2000
0.0005
0.0010
0.0020
False Positive Identification Rate
T
L
F
0.0005
0.0010
0.0020
0.5000
U
M
G
Figure 96: DETs from the second submission for all participants in all classes.
Q
J
D
0.0050
0.0100
0.0200
0.5000
0.5000
0.0500
0.1000
0.2000
0.0005
0.0010
0.0020
V
O
H
Ten−Finger Plain−to−Rolled (2)
0.0050
0.0100
0.0200
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
0.0500
0.1000
0.2000
Right Slap (2)
Left Slap (2)
Left and Right Slap (2)
0.5000
Left Index (2)
Right Index (2)
Left and Right Index (2)
FPVTE – F INGERPRINT M ATCHING
111
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
False Negative Identification Rate
FPVTE – F INGERPRINT M ATCHING
112
Miss Rate
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
0.2000
0.1000
0.0500
0.0200
0.0100
0.0050
0.0020
0.0010
E
L
F
M
G
O
H
Ten−Finger Plain−to−Rolled (1)
D
K
Identification Flats (1)
Ten−Finger Plain−to−Plain (1)
Ten−Finger Rolled−to−Rolled (1)
C
J
Right Slap (1)
Left Slap (1)
Left and Right Slap (1)
I
15
V
10
U
5
T
15
S
10
Q
5
P
10
15
Left Index (1)
Right Index (1)
Left and Right Index (1)
5
Rank
Figure 97: CMCs from the first submission for all participants in all classes.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
0.0010
0.0020
0.0050
0.0100
0.0200
0.0500
0.1000
0.2000
5
10
P
I
C
15
Left Index (2)
Right Index (2)
Left and Right Index (2)
5
10
S
K
E
15
Rank
T
L
F
Identification Flats (2)
Ten−Finger Plain−to−Plain (2)
Ten−Finger Rolled−to−Rolled (2)
5
10
U
M
G
15
V
O
H
Ten−Finger Plain−to−Rolled (2)
Figure 98: CMCs from the second submission for all participants in all classes.
Q
J
D
Right Slap (2)
Left Slap (2)
Left and Right Slap (2)
FPVTE – F INGERPRINT M ATCHING
113
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
Miss Rate
114
C
FPVTE – F INGERPRINT M ATCHING
Accuracy Time Tradeoff Detailed Tables with Median Values
In order to reduce the number of tables in the main body of the report (Section 8), this appendix contains tables that show
the search times for each stage of identification, for both a single process and ten processes.
The tables in this appendix report median times. For readers interested in mean times, please refer to Appendix D.
Class A results are in Tables 24 through 29, Class B results are in Tables 30 through 33, and Class C results are in Tables 34
through 36.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.76
3.89
0.35
16.90
2.49
3.56
9.74
3.61
4.13
9.94
6
17
3
30
12
15
23
16
19
24
1
1
2
1
1
1
1
1
1.56
1.32
3.33
11.25
7.67
16.86
3.99
5.97
1.41
6.01
10
8
14
28
22
29
18
20
9
21
2
1
2
1
1
1
1
1
V
2
2
0.62
1
5
2
1
3.32
13
1
10.70
0.28
2
2
2
10.32
25
1
1.84
10.44
26
2
27
1.25
7
11
0.54
4
2
2
2
0.26
One
1
1
Sub. #
U
T
S
Q
P
O
M
L
K
J
I
H
G
F
E
D
C
Letter
20
7
21
19
30
22
28
13
9
10
5
23
11
15
2
26
27
8
4
25
18
17
24
14
12
29
3
16
6
1
Stage One
5.81
1.14
7.15
5.43
17.66
7.73
11.61
3.20
1.36
1.62
0.61
9.58
1.78
3.79
0.27
11.18
11.21
1.28
0.56
10.36
5.09
4.54
10.10
3.52
2.40
16.93
0.38
3.90
0.72
0.26
Ten
1
30
1
1
1
27
29
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
1
1
—
12.96
—
—
—
2.64
4.48
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
3.54
—
—
One
1
30
1
1
1
28
29
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
27
1
1
Stage Two
—
—
—
17.63
—
—
—
1.70
1.79
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
1.62
Ten
19
27
18
16
29
24
28
13
8
9
5
26
10
12
2
23
25
7
4
22
17
15
21
14
11
30
3
20
6
1
6.01
14.42
5.97
3.99
16.86
10.43
15.70
3.33
1.32
1.56
0.62
10.70
1.84
3.32
0.29
10.32
10.44
1.25
0.54
9.94
4.13
3.61
9.74
3.56
2.49
16.90
0.35
7.32
0.76
0.26
One
19
30
20
17
29
21
27
12
8
9
5
22
10
14
2
25
26
7
4
24
16
15
23
13
11
28
3
18
6
1
Total
5.81
18.76
7.15
5.44
17.66
9.45
13.44
3.20
1.36
1.62
0.61
9.58
1.78
3.79
0.27
11.18
11.21
1.28
0.56
10.36
5.09
4.54
10.10
3.52
2.40
16.93
0.38
5.56
0.72
0.26
Ten
0.0226
0.0571
NA
0.0685
0.1218
0.0253
7
30
9
21
4
0.0222
3
0.1272
2
0.1308
23
22
0.0766
0.0818
15
13
0.2921
0.2995
28
0.0351
6
0.0625
8
29
0.0875
0.0883
17
16
0.0712
0.0786
10
0.0257
5
14
0.1576
26
0.1607
0.1089
19
27
0.1082
0.1111
18
0.0723
0.0745
12
20
0.0197
1
11
0.1337
0.1335
25
24
FNIR @ FPIR = 10−3
Table 24: Tabulation of median identification time results for Class A — Left Index — less than 20-second searches. Letter refers to the participant’s letter code found on the footer of this page.
Sub. # is an identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with
Total indicating the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median durations are reported in
seconds, but were originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. The number to the left of a value provides
the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 7 are reprinted to the right of this table.
< 20 seconds
Participant
FPVTE – F INGERPRINT M ATCHING
115
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
116
6
—
22.44
1
1
6
—
—
—
3.14
1
4
2
5
3
65.35
24.16
44.66
40.99
54.03
41.84
6
2
4
3
5
1
64.02
39.70
47.82
43.35
56.40
21.60
2
6
4
3
5
1
0.0252
0.1178
0.0650
0.0278
0.1086
0.0197
FNIR @ FPIR = 10−3
18.37
1
—
1
0.35
6
Total
1
56.40
1
—
5
—
Stage Two
18.19
5
43.35
1
0.41
1
Stage One
Sub. #
1
54.03
3
47.82
5
—
Participant
Letter
2
5
40.99
4
39.38
1
Ten
D
2
3
44.66
2
64.02
One
G
2
4
23.74
6
Ten
I
2
2
65.35
One
S
2
6
Ten
U
2
One
V
Table 25: Tabulation of median identification time results for Class A — Left Index — greater than or equal to 20-second searches. Letter refers to the participant’s letter code found on the
footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined
in Section 5, with Total indicating the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median durations
are reported in seconds, but were originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. The number to the left of
a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 7 are reprinted to the right of
this table.
≥ 20 seconds
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.69
3.77
0.32
16.27
2.38
3.60
10.10
3.56
4.05
9.83
6
18
3
29
12
16
25
15
19
24
1
1
2
1
1
1
1
1
1.36
1.24
3.07
10.94
7.39
16.55
3.73
5.86
1.24
5.60
10
9
13
28
22
30
17
21
8
20
2
1
2
1
1
1
1
1
V
2
2
0.56
1
5
2
1
3.26
14
1
9.39
0.26
2
2
2
10.26
26
1
1.78
10.42
27
2
23
1.13
7
11
0.51
4
2
2
2
0.24
One
1
1
Sub. #
U
T
S
Q
P
O
M
L
K
J
I
H
G
F
E
D
C
Letter
19
7
21
20
30
22
28
13
9
10
5
23
11
16
2
27
26
8
4
24
18
17
25
14
12
29
3
15
6
1
Stage One
5.61
1.01
7.21
5.66
16.75
7.57
11.33
3.20
1.28
1.53
0.58
8.24
1.64
3.71
0.27
10.91
10.81
1.20
0.53
9.86
5.05
4.58
10.23
3.36
2.34
16.58
0.35
3.68
0.68
0.24
Ten
1
30
1
1
1
27
28
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
29
1
1
—
9.19
—
—
—
2.53
3.24
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
3.69
—
—
One
1
30
1
1
1
28
29
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
27
1
1
Stage Two
—
—
—
14.92
—
—
—
1.67
1.73
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
1.56
Ten
18
27
19
16
30
23
28
12
8
9
5
21
10
13
2
25
26
7
4
22
17
14
24
15
11
29
3
20
6
1
5.60
10.76
5.87
3.73
16.55
9.98
14.24
3.07
1.24
1.36
0.56
9.39
1.78
3.26
0.26
10.27
10.42
1.13
0.51
9.83
4.05
3.56
10.10
3.60
2.38
16.27
0.32
7.46
0.69
0.24
One
18
28
20
19
30
22
27
12
8
9
5
21
10
14
2
26
25
7
4
23
16
15
24
13
11
29
3
17
6
1
Total
5.61
15.85
7.21
5.66
16.75
9.24
13.09
3.20
1.28
1.53
0.58
8.24
1.64
3.71
0.27
10.91
10.81
1.20
0.53
9.86
5.06
4.59
10.23
3.36
2.34
16.58
0.35
5.27
0.68
0.24
Ten
0.0996
0.0223
21
5
0.1929
28
0.0562
0.0442
7
9
0.0214
0.0218
2
0.1100
4
0.1133
25
22
0.0675
0.0776
17
13
0.2526
0.2615
29
0.0295
6
0.0505
8
30
0.0685
0.0682
14
15
0.0643
0.0708
12
0.0215
3
16
0.1230
26
0.1249
0.0910
19
27
0.0903
0.0933
18
0.0624
0.0630
11
20
0.0190
1
10
0.1124
0.1132
23
24
FNIR @ FPIR = 10−3
Table 26: Tabulation of median identification time results for Class A — Right Index — less than 20-second searches. Letter refers to the participant’s letter code found on the footer of this
page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5,
with Total indicating the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median durations are reported in
seconds, but were originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. The number to the left of a value provides
the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 8 are reprinted to the right of this table.
< 20 seconds
Participant
FPVTE – F INGERPRINT M ATCHING
117
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
118
6
—
11.53
1
1
6
—
—
—
3.02
1
4
3
5
2
61.24
20.10
46.05
40.35
59.55
28.64
6
2
4
3
5
1
60.53
34.54
45.80
42.08
58.30
20.55
3
6
4
2
5
1
0.0222
0.1007
0.0503
0.0214
0.0909
0.0190
FNIR @ FPIR = 10−3
17.53
1
—
1
0.32
6
Total
1
58.30
1
—
5
—
Stage Two
17.43
5
42.08
1
0.33
1
Stage One
Sub. #
1
59.55
3
45.80
5
—
Participant
Letter
2
5
40.35
4
34.25
1
Ten
D
2
3
46.05
2
60.53
One
G
2
4
19.74
6
Ten
I
2
2
61.24
One
S
2
6
Ten
U
2
One
V
Table 27: Tabulation of median identification time results for Class A — Right Index — greater than or equal to 20-second searches. Letter refers to the participant’s letter code found on the
footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined
in Section 5, with Total indicating the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median durations
are reported in seconds, but were originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. The number to the left of
a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 8 are reprinted to the right of
this table.
≥ 20 seconds
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
< 20 seconds
1
1
1
1
1
1
I
J
K
L
O
V
2.08
6.35
5.22
17.87
13.64
18.01
2.19
14.46
9.29
1
4
3
8
6
9
2
7
5
One
5
7
2
9
6
8
4
3
1
Stage One
9.45
15.50
5.05
18.95
14.42
18.87
6.62
6.54
2.23
Ten
1
1
1
1
1
1
1
1
1
—
—
—
—
—
—
—
—
—
One
1
1
1
1
1
1
1
1
1
Stage Two
—
—
—
—
—
—
—
—
—
Ten
5
7
2
9
6
8
3
4
1
9.29
14.46
2.19
18.01
13.64
17.87
5.22
6.35
2.08
One
5
7
2
9
6
8
4
3
1
Total
9.45
15.50
5.05
18.95
14.42
18.87
6.62
6.54
2.23
Ten
0.0374
0.0515
0.0058
0.0143
0.0360
0.0146
0.0229
0.0034
9
2
3
6
4
5
1
0.0368
8
7
FNIR @ FPIR = 10−3
Table 28: Tabulation of median identification time results for Class A — Left and Right Index — less than 20-second searches. Letter refers to the participant’s letter code found on the footer of
this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5,
with Total indicating the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median durations are reported in
seconds, but were originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. The number to the left of a value provides
the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 9 are reprinted to the right of this table.
1
2
1
Sub. #
G
C
Letter
Participant
FPVTE – F INGERPRINT M ATCHING
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
119
FPVTE – F INGERPRINT M ATCHING
120
15
237.43
70.99
27
1
21
6
18.25
182.69
37.59
10
11
4
4
0.0202
0.0207
0.0030
0.0030
FNIR @ FPIR = 10−3
23
16.35
Ten
19.23
1
Total
24
7.29
27
One
52.34
23
—
Ten
24
68.09
1
Stage Two
19.30
26
1
One
2
169.73
—
Ten
19.27
21
1
Stage One
2
164.19
1
One
1
21
18.25
Sub. #
2
1
0.0412
27
22
0.0311
16.35
15
0.0686
1
68.04
24
0.0684
27
13
233.71
23
0.0030
1
22
39.83
4
H
M
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
Participant
Letter
D
E
73.95
7
54.92
0.0386
16
221.16
10
21
22
36.71
24
57.10
—
8
52.25
0.0143
512.28
1
—
12
8
11
1
—
25
36.28
377.31
0.0286
60.66
—
1
—
5
14
0.0072
518.11
1
—
1
33.35
338.88
7
NA
13
1
—
1
7
33.62
26
NA
—
68.04
1
—
—
4
69.95
25
0.0214
—
13
233.70
1
—
32.93
14
30.23
12
1
23
39.83
1
1
6
23.42
3
169.90
—
73.95
8
54.92
—
3
31.44
8
19
—
16
221.16
10
—
—
5
171.24
1
1
23
36.71
24
1
1
—
20
57.10
2
10
52.25
36.28
377.30
—
1
—
10
512.28
2
13
7
1
—
1
—
11
1
25
33.35
338.88
33.62
1
—
1
60.66
2
9
6
69.95
1
1
518.11
2
32.93
14
30.23
—
14
2
8
23.42
5
1
2
J
I
2
4
31.44
1
F
K
2
7
169.90
G
L
1
9
22
0.0333
171.24
16
0.0027
22
1
0.0027
12
105.73
1
0.0281
2
16
242.28
13
0.0195
0.0370
23
175.09
9
20
101.16
20
22.50
46.77
17
212.69
2
62.81
21
161.02
26
12
—
19
23.00
43.53
1
80.55
2
63.65
26
83.71
26
14
—
27
—
—
1
57.36
1
—
25
69.98
1
1
105.73
27
—
—
17
160.28
1
—
20
89.06
1
1
101.16
16
22.50
46.77
18
153.61
3
62.81
1
20
86.83
26
12
2
17
23.00
43.53
1
3
NA
63.65
2
26
27
15
1
88.13
0.0366
487.65
19
0.0336
15
131.08
17
0.0358
25.68
18
52.79
18
0.0028
495.50
37.23
9
407.99
3
4
9
45.51
25
128.23
—
—
11
240.40
17
—
1
25.27
24
127.65
1
—
25
0.63
18
—
1
17.03
22
—
—
131.01
23
0.68
1
1
19
27.53
22
—
88.07
37.23
4
407.47
1
487.65
1
11
28.49
25
128.23
15
2
6
239.62
18
25.68
1
24
127.65
495.50
2
19
5
2
2
2
O
P
Q
S
T
U
V
Table 29: Tabulation of median identification time results for Class A — Left and Right Index — greater than or equal to 20-second searches. Letter refers to the participant’s letter code found
on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification
defined in Section 5, with Total indicating the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median
durations are reported in seconds, but were originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. The number to
the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 9 are reprinted to the
right of this table.
≥ 20 seconds
50.78
71.69
71.77
33.57
42.79
36.07
47.97
21
26
27
10
15
12
18
2
1
2
1
2
1
2
50.00
20
1
1
73.15
2
28
74.38
29
1
2
25.61
7
2
37.01
38.26
14
1
13
55.96
24
2
1
51.77
22
1
87.21
49.75
19
2
30
59.49
25
1
2
16.24
6
2
31.17
43.59
16
1
9
30.09
8
2
1
34.37
11
1
5.56
2.82
1
12.37
52.89
23
2
5
47.93
17
1
2
6.74
4
2
3
3.74
One
2
1
Sub. #
NA
—
15
50.36
36.78
NA
—
11
74.84
74.67
53.59
54.90
77.48
39.34
92.65
33.44
30.25
10.01
78.75
27.09
42.90
66.74
60.07
61.05
60.00
17.59
47.82
32.85
35.02
3.19
80.04
70.13
7.03
3.88
Ten
24
23
16
17
25
12
28
9
7
4
26
6
13
21
19
20
18
5
14
8
10
1
27
22
3
2
Stage One
1
1
29
30
1
1
27
26
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
25
1
1
—
—
45.59
47.07
—
—
3.55
3.49
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
4.05
3.29
—
—
One
1
1
—
—
1
1
26
25
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
27
1
1
Stage Two
—
—
—
—
NA
NA
—
—
3.32
3.20
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
9.04
6.08
Ten
15
11
30
28
25
24
20
19
26
12
29
9
5
3
27
7
13
21
17
16
23
6
14
8
10
1
22
18
4
2
47.97
36.07
89.07
80.03
71.77
71.69
54.02
53.24
73.15
37.01
87.21
31.17
12.37
5.56
74.38
25.61
38.26
55.96
51.77
49.75
59.49
16.24
43.59
30.09
34.37
2.82
58.10
52.23
6.74
3.74
One
15
11
—
—
23
22
16
17
25
12
28
9
7
4
26
6
13
21
19
20
18
5
14
8
10
1
27
24
3
2
Total
50.37
36.78
NA
NA
74.84
74.67
57.02
58.21
77.48
39.34
92.65
33.44
30.25
10.01
78.75
27.09
42.90
66.74
60.07
61.05
60.00
17.59
47.82
32.85
35.02
3.19
89.04
76.64
7.03
3.88
Ten
0.0654
0.0647
0.0163
0.0142
0.0259
0.0187
0.1684
0.1681
0.0371
0.0325
0.0998
0.1008
0.0116
0.0094
0.0287
0.0236
0.0288
0.0276
0.1736
0.1634
0.0257
0.0254
0.0098
0.0099
0.1089
0.1133
0.0500
0.0461
0.0192
0.0190
22
21
6
5
13
7
29
28
18
17
23
24
4
1
15
10
16
14
30
27
12
11
2
3
25
26
20
19
9
8
FNIR @ FPIR = 10−3
Table 30: Tabulation of median identification time results for Class B — Left Slap. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to
differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating the combined
search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median durations are reported in seconds, but were originally
recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the value could
not be performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR
values from Table 10 are reprinted to the right of this table.
V
U
S
Q
O
M
L
J
I
H
G
F
E
D
C
Letter
Participant
FPVTE – F INGERPRINT M ATCHING
121
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
122
—
4
2
53.32
6.50
3.67
24
3
2
76.16
6.78
3.75
13
2
6
23
24
0.0151
0.0052
0.0072
0.0392
0.0403
FNIR @ FPIR = 10−3
1
—
19
92.10
Total
—
1
5.79
1
27
Stage Two
1
—
27
57.19
Stage One
3.74
1
3.56
1
21
Ten
2
6.78
26
8.74
One
3.67
3
69.48
1
28
Ten
2
6.50
22
4.41
One
1
4
48.43
1
28
Ten
2
18
82.56
One
1
1
27
0.1222
52.47
29
0.1220
1
28
0.0212
21
33.38
18
0.0198
2
8
48.57
16
0.0641
0.0083
15
10.56
25
7
31.87
4
39.25
3.23
8
45.92
12
34.32
15
10.13
18
10
—
5
37.28
2.97
1
—
13
34.47
1
—
17
10
—
1
—
—
1
—
1
—
1
—
1
1
33.38
1
—
—
8
48.57
1
—
15
10.56
1
1
31.87
4
39.25
3.23
8
45.92
12
34.32
2
17
10.13
18
10
1
5
37.28
2.97
2
14
0.0647
34.47
1
20
26
11
2
61.42
0.0058
60.56
5
0.0045
17
66.70
1
0.0156
50.16
20
48.17
14
0.0126
52.84
55.87
14
26.83
10
18
20
41.13
6
80.24
—
—
14
24.14
26
—
1
—
7
74.09
1
—
1
—
27
—
1
—
1
—
—
66.70
1
—
1
0.0167
1
21
48.17
1
—
15
61.42
55.87
14
26.83
1
5
60.56
2
23
41.13
6
80.24
3
17
1
16
24.14
26
1
50.16
2
7
74.08
1
0.0202
52.84
1
29
5
17
22
2
3
10.64
0.1259
31.81
30
0.1155
7
34.25
27
0.0142
5.47
9
92.89
12
0.0132
12.66
32.03
28
39.42
11
6
9
87.41
13
78.87
—
—
29
35.61
25
—
1
—
12
73.93
1
—
1
—
26
—
1
—
1
—
—
34.25
1
—
1
0.0057
1
9
92.89
1
—
3
10.64
32.03
28
39.42
1
21
31.80
2
9
87.41
13
78.87
23
7
1
30
35.61
25
25
5.47
2
13
73.93
27
12.66
1
28
20
6
2
25
0.0369
0.0057
21
0.0381
3
22
0.0266
69.69
74.64
19
0.0273
66.61
22
75.02
20
0.0106
19
70.54
23
NA
8
0.0110
64.04
24
70.84
—
NA
9
61.94
—
25
89.26
—
37.08
22
1
—
30
80.11
11
50.24
3.25
—
1
NA
28
35.43
16
3.30
1
—
—
NA
11
48.51
26
74.64
1
54.61
—
—
16
3.65
23
75.02
30
40.70
1
—
3.54
70.54
24
NA
29
—
1
25
2
26
70.84
—
NA
1
—
66.42
1
27
34.38
—
37.08
1
63.34
2
10
40.66
11
50.24
19
1
15
35.43
16
60.64
2
12
48.51
58.39
1
19
24
2
1
1
1
1
Sub. #
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Table 31: Tabulation of median identification time results for Class B — Right Slap. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to
differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating the combined
search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median durations are reported in seconds, but were originally
recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the value could
not be performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR
values from Table 11 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
52.39
83.33
82.13
29.82
38.83
38.93
52.79
20
30
29
10
16
17
21
2
1
2
1
2
1
2
63.79
24
1
1
68.46
2
25
69.28
26
1
2
26.15
7
2
38.34
46.13
19
1
15
37.19
14
2
1
73.06
28
1
45.91
70.38
27
2
18
34.55
12
1
2
3.89
1
2
27.15
36.36
13
1
8
27.27
9
2
1
33.09
11
1
20.78
6.12
2
10.47
60.79
23
2
6
59.32
22
1
2
10.70
5
2
4
6.40
One
3
1
Sub. #
NA
—
17
55.29
41.09
NA
—
12
85.11
85.38
57.81
69.86
73.17
41.51
51.07
30.00
63.81
22.48
74.42
28.45
54.07
42.38
85.16
86.18
34.99
4.17
40.13
30.15
35.61
6.09
106.49
104.95
11.10
6.63
Ten
23
25
18
20
21
13
15
7
19
5
22
6
16
14
24
26
9
1
11
8
10
2
28
27
4
3
Stage One
1
1
29
30
1
1
25
26
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
27
1
1
—
—
43.88
53.54
—
—
1.05
1.07
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
2.28
1.34
—
—
One
1
1
—
—
1
1
25
26
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
27
1
1
Stage Two
—
—
—
—
NA
NA
—
—
0.76
0.79
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
4.43
2.40
Ten
18
15
28
30
27
29
19
22
23
14
16
8
6
4
24
7
17
13
26
25
11
1
12
9
10
2
21
20
5
3
52.80
38.93
82.40
83.62
82.13
83.33
53.40
65.02
68.46
38.34
45.91
27.15
20.78
10.48
69.28
26.15
46.13
37.19
73.06
70.38
34.55
3.89
36.36
27.27
33.09
6.12
63.05
61.24
10.70
6.40
One
17
12
—
—
23
25
18
20
21
13
15
7
19
5
22
6
16
14
24
26
9
1
11
8
10
2
28
27
4
3
Total
55.29
41.09
NA
NA
85.11
85.38
58.57
70.73
73.17
41.51
51.07
30.01
63.82
22.48
74.42
28.45
54.07
42.38
85.16
86.18
34.99
4.17
40.13
30.15
35.61
6.09
111.35
108.01
11.10
6.63
Ten
NA
NA
0.0031
0.0024
0.0063
0.0049
0.0910
0.0901
0.0106
0.0084
0.0349
0.0361
0.0022
0.0015
0.0068
0.0047
0.0054
0.0062
0.0904
0.0882
0.0057
0.0051
0.0021
0.0022
0.0160
0.0190
0.0139
0.0124
0.0036
0.0036
30
29
6
5
15
10
28
26
18
17
23
24
3
1
16
9
12
14
27
25
13
11
2
3
21
22
20
19
7
7
FNIR @ FPIR = 10−3
Table 32: Tabulation of median identification time results for Class B — Left and Right Slap. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier
used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating the
combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median durations are reported in seconds, but were
originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the
value could not be performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference,
the FNIR values from Table 12 are reprinted to the right of this table.
V
U
S
Q
O
M
L
J
I
H
G
F
E
D
C
Letter
Participant
FPVTE – F INGERPRINT M ATCHING
123
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
124
—
4
3
45.15
10.21
7.92
22
4
3
80.68
10.62
7.99
16
2
6
29
30
0.0043
0.0012
0.0020
NA
NA
FNIR @ FPIR = 10−3
1
—
17
86.69
Total
—
1
1.33
2
24
Stage Two
1
—
27
46.70
Stage One
7.99
1
0.86
2
18
Ten
3
10.62
25
2.30
One
7.92
4
79.06
1
28
Ten
3
10.21
22
1.43
One
1
4
44.48
1
26
Ten
2
19
83.76
One
1
2
23
0.0591
45.79
27
0.0591
2
27
0.0062
20
40.93
18
0.0040
2
11
54.18
14
0.0203
0.0024
17
4.52
23
7
37.56
1
14.99
7.24
12
49.34
5
48.02
20
4.26
27
13
—
1
16.26
6.76
1
—
6
43.61
1
—
26
15
—
1
—
—
1
—
1
—
1
—
1
1
40.93
1
—
—
11
54.18
1
—
17
4.52
1
1
37.56
1
14.99
7.23
13
49.34
5
48.02
2
22
4.26
27
13
1
1
16.26
6.76
2
6
0.0204
43.61
1
27
24
17
2
97.33
0.0012
98.56
2
0.0009
28
21.52
1
0.0049
82.66
6
54.10
17
0.0033
86.56
19.07
16
26.88
11
27
7
43.86
7
64.19
—
—
16
25.38
19
—
1
—
8
60.13
1
—
1
—
22
—
1
—
1
—
—
21.52
1
—
1
0.0031
1
6
54.10
1
—
10
97.33
19.07
16
26.88
1
8
98.56
2
7
43.86
7
64.19
5
28
1
18
25.38
19
1
82.66
2
8
60.13
1
0.0033
86.56
1
24
8
11
28
2
5
34.39
0.0543
86.02
26
0.0515
23
42.78
25
0.0041
14.50
12
72.75
15
0.0035
28.59
38.87
20
38.07
13
9
13
65.91
10
63.40
—
—
23
35.64
18
—
1
—
11
60.02
1
—
1
—
21
—
1
—
1
—
—
42.78
1
—
1
0.0012
1
12
72.75
1
—
2
34.38
38.87
20
38.07
1
14
86.01
2
14
65.91
10
63.40
14
24
1
26
35.64
18
25
14.49
2
12
60.02
28
28.59
1
23
14
9
2
16
0.0108
0.0012
20
0.0136
2
21
0.0099
50.75
90.62
19
0.0141
79.70
25
92.93
22
0.0027
21
87.60
26
NA
9
0.0024
42.92
29
88.46
—
NA
7
66.01
—
30
86.60
—
37.51
24
1
—
28
75.28
9
51.13
1.00
—
1
NA
25
34.96
15
1.02
1
—
—
NA
10
49.30
26
90.62
1
57.06
—
—
19
1.48
25
92.93
30
35.79
1
—
1.48
87.60
26
NA
29
—
1
27
2
29
88.46
—
NA
1
—
49.75
1
30
29.91
—
37.51
1
78.68
2
10
41.20
9
51.13
21
1
15
34.96
15
41.50
2
11
49.30
64.62
1
21
25
2
1
1
1
1
Sub. #
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Table 33: Tabulation of median identification time results for Class B — Identification Flats. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier
used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating the
combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median durations are reported in seconds, but were
originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the
value could not be performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference,
the FNIR values from Table 13 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
33.71
86.56
91.74
47.96
51.07
47.02
62.26
8
29
30
15
16
14
21
2
1
2
1
2
1
2
35.17
9
1
1
73.41
2
27
77.75
28
1
2
43.57
12
2
68.54
38.28
10
1
24
52.41
18
2
1
65.36
22
1
46.70
68.07
2
13
23.69
7
23
1
2
7.71
1
2
43.37
60.86
20
1
11
51.69
17
2
1
52.91
19
1
20.73
12.77
2
17.50
71.36
25
2
6
72.26
26
1
2
18.38
5
2
3
18.30
One
4
1
Sub. #
NA
—
18
65.76
48.95
NA
—
13
98.32
95.71
35.06
38.39
83.44
74.17
46.86
43.13
66.62
39.57
84.93
47.12
40.63
63.47
73.09
78.40
30.21
8.79
61.46
51.98
54.82
13.68
137.48
141.45
19.05
19.19
Ten
26
25
6
7
23
21
11
10
19
8
24
12
9
17
20
22
5
1
16
14
15
2
27
28
3
4
Stage One
1
1
29
30
1
1
28
27
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
26
25
1
1
—
—
17.32
26.89
—
—
1.70
1.63
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
1.63
1.36
—
—
One
1
1
—
—
1
1
25
26
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
27
1
1
Stage Two
—
—
—
—
NA
NA
—
—
1.36
1.51
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
2.10
1.85
Ten
19
14
22
27
30
29
8
9
24
23
13
11
6
3
28
12
10
16
20
21
7
1
18
15
17
2
26
25
5
4
62.26
47.02
68.30
74.94
91.74
86.56
35.35
36.86
73.41
68.54
46.70
43.37
23.53
17.51
77.75
43.57
38.28
52.41
65.36
68.07
23.69
7.71
60.86
51.69
52.91
12.77
74.36
73.93
18.38
18.30
One
18
13
—
—
26
25
6
8
23
20
11
10
21
7
24
12
9
17
19
22
5
1
16
14
15
2
27
28
3
4
Total
65.76
48.95
NA
NA
98.32
95.71
36.23
39.96
83.44
74.17
46.86
43.13
74.43
39.58
84.93
47.12
40.63
63.47
73.10
78.40
30.21
8.79
61.46
51.98
54.82
13.68
140.57
143.53
19.05
19.19
Ten
NA
0.0711
0.0015
0.0011
0.0088
0.0048
0.0734
0.0734
0.0368
0.0276
0.0276
0.0275
0.0013
0.0010
0.0047
0.0027
0.0102
0.0095
0.0934
0.0826
0.0025
0.0027
0.0011
0.0013
0.0311
0.1680
0.0163
0.0155
0.0024
0.0024
30
24
6
2
14
13
25
25
23
20
20
19
4
1
12
10
16
15
28
27
9
10
2
4
22
29
18
17
7
7
FNIR @ FPIR = 10−3
Table 34: Tabulation of median identification time results for Class C — Ten-Finger Plain-to-Plain. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an
identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating
the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median durations are reported in seconds, but were
originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the
value could not be performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference,
the FNIR values from Table 14 are reprinted to the right of this table.
V
U
S
Q
O
M
L
J
I
H
G
F
E
D
C
Letter
Participant
FPVTE – F INGERPRINT M ATCHING
125
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
126
—
6
4
58.92
21.97
20.15
25
5
4
83.71
22.27
21.10
18
7
4
15
16
0.0106
0.0018
0.0015
0.0085
0.0094
FNIR @ FPIR = 10−3
1
—
17
128.86
Total
—
1
1.40
2
28
Stage Two
1
—
25
74.97
Stage One
21.10
1
2.42
2
27
Ten
4
22.27
28
1.79
One
20.15
5
81.93
1
26
Ten
5
21.97
25
1.86
One
1
6
56.37
1
25
Ten
2
19
126.56
One
1
2
28
0.0536
73.52
25
0.0536
2
25
0.0447
28
59.83
24
0.0333
2
13
72.22
21
0.0201
0.0050
19
7.67
20
12
55.79
1
18.47
9.82
16
68.20
3
44.54
20
7.78
26
9
—
1
19.46
8.63
1
—
3
42.73
1
—
30
11
—
1
—
—
1
—
1
—
1
—
1
1
59.83
1
—
—
14
72.22
1
—
20
7.67
1
1
55.79
1
18.47
9.82
18
68.20
3
44.54
2
22
7.78
26
9
1
1
19.46
8.63
2
3
0.0199
42.73
1
30
19
11
2
94.60
0.0013
95.02
1
0.0014
27
56.96
2
0.0051
84.51
12
30.34
13
0.0033
84.50
54.56
6
36.39
9
29
15
30.82
7
75.20
—
—
7
33.02
23
—
1
—
9
67.98
1
—
1
—
19
—
1
—
1
—
—
56.96
1
—
1
0.0097
1
12
30.34
1
—
17
94.60
54.56
6
36.39
1
15
95.02
2
17
30.82
7
75.20
8
27
1
7
33.02
23
1
84.51
2
9
67.98
1
0.0083
84.50
1
21
16
14
29
2
8
64.81
0.0783
66.39
28
0.0716
16
50.93
27
0.0034
31.68
10
52.10
11
0.0033
20.25
48.08
11
60.77
9
5
12
48.80
14
75.08
—
—
13
54.42
22
—
1
—
14
69.30
1
—
1
—
21
—
1
—
1
—
—
50.93
1
—
1
0.0017
1
10
52.10
1
—
5
64.80
48.08
11
60.77
1
21
57.57
2
12
48.80
15
75.08
26
13
1
14
54.41
22
27
31.68
2
16
69.30
26
19.64
1
23
19
4
2
27
0.0860
0.0014
29
0.2462
2
30
0.0358
73.83
77.04
23
0.0351
69.95
24
72.56
22
0.0017
18
69.71
20
NA
5
0.0019
74.40
22
70.77
—
NA
8
74.22
—
23
82.61
—
42.25
25
1
—
28
72.48
8
67.33
2.49
—
1
NA
24
40.74
17
2.51
1
—
—
NA
10
65.07
28
77.04
1
34.90
—
—
18
2.22
24
72.56
30
21.02
1
—
2.27
69.71
21
NA
29
—
1
27
2
24
70.77
—
NA
1
—
71.41
1
25
48.12
—
42.25
1
67.31
2
13
52.49
8
67.32
17
1
15
40.74
18
72.31
2
10
65.07
72.05
1
20
26
2
1
1
1
1
Sub. #
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Table 35: Tabulation of median identification time results for Class C — Ten-Finger Rolled-to-Rolled. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an
identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating
the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median durations are reported in seconds, but were
originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the
value could not be performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference,
the FNIR values from Table 15 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
37.74
88.48
88.07
48.76
53.93
48.40
69.87
10
30
29
15
17
14
28
2
1
2
1
2
1
2
36.83
9
1
1
66.80
2
26
67.88
27
1
2
38.41
11
2
59.05
41.48
13
1
22
52.47
16
2
1
65.74
25
1
57.91
64.83
2
20
22.14
4
24
1
2
16.78
2
2
57.63
61.87
23
1
19
58.57
21
2
1
34.06
7
1
18.30
8.99
1
27.47
56.17
18
2
3
41.16
12
1
2
34.85
8
2
5
33.68
One
6
1
Sub. #
NA
—
22
73.98
50.40
NA
—
12
92.90
95.29
36.93
39.07
74.41
64.96
59.52
58.09
48.54
58.46
75.26
41.85
45.62
67.66
73.89
77.97
26.40
20.00
62.74
60.32
35.00
9.76
109.60
69.81
36.23
35.85
Ten
26
27
7
8
23
18
15
13
11
14
24
9
10
19
21
25
3
2
17
16
4
1
28
20
6
5
Stage One
1
1
29
30
1
1
27
28
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
26
25
1
1
—
—
20.09
33.16
—
—
1.89
1.91
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
1.04
0.60
—
—
One
1
1
—
—
1
1
28
27
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
26
25
1
1
Stage Two
—
—
—
—
NA
NA
—
—
2.36
1.70
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
1.16
0.58
Ten
26
14
27
28
29
30
10
11
24
20
18
17
3
5
25
9
12
15
23
22
4
2
21
19
7
1
16
13
8
6
69.87
48.40
74.56
82.88
88.07
88.48
39.37
39.76
66.80
59.05
57.91
57.63
20.18
27.47
67.88
38.41
41.48
52.47
65.74
64.83
22.14
16.78
61.87
58.57
34.06
8.99
57.36
41.67
34.85
33.68
One
22
11
—
—
26
27
7
8
23
18
14
12
16
13
24
9
10
19
21
25
3
2
17
15
4
1
28
20
6
5
Total
73.98
50.40
NA
NA
92.90
95.29
39.44
40.80
74.41
64.96
59.52
58.09
60.35
58.47
75.26
41.85
45.62
67.66
73.89
77.97
26.40
20.00
62.74
60.32
35.00
9.76
110.64
70.49
36.23
35.85
Ten
0.0149
0.0169
0.0028
0.0018
0.0137
0.0056
0.2514
0.2514
0.0649
0.0521
0.0291
0.0285
0.0014
0.0011
0.0071
0.0034
0.0136
0.0129
0.3067
0.2514
0.0041
0.0036
0.0020
0.0022
0.1017
0.2366
0.0378
0.0295
0.0052
0.0039
17
18
6
3
16
12
27
27
24
23
20
19
2
1
13
7
15
14
30
27
10
8
4
5
25
26
22
21
11
9
FNIR @ FPIR = 10−3
Table 36: Tabulation of median identification time results for Class C — Ten-Finger Plain-to-Rolled. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an
identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating
the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are median durations are reported in seconds, but were
originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the
value could not be performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference,
the FNIR values from Table 16 are reprinted to the right of this table.
V
U
S
Q
O
M
L
J
I
H
G
F
E
D
C
Letter
Participant
FPVTE – F INGERPRINT M ATCHING
127
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
128
D
FPVTE – F INGERPRINT M ATCHING
Accuracy Time Tradeoff Detailed Tables with Mean Values
In order to reduce the number of tables in the main body of the report (Section 8), this appendix contains tables that show
the search times for each stage of identification, for both a single process and ten processes.
The tables in this appendix report mean times. For readers interested in median times, please refer to Appendix C.
Class A results are in Tables 37 through 42, Class B results are in Tables 43 through 46, and Class C results are in Tables 47
through 49.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.87
3.87
0.57
16.71
2.52
3.78
9.52
3.65
4.26
10.36
6
17
4
30
12
16
23
15
19
24
1
1
2
1
1
1
1
1
1.63
1.38
3.43
11.44
7.71
15.37
4.18
5.96
1.45
6.22
10
8
14
27
22
29
18
20
9
21
2
1
2
1
1
1
1
1
V
2
2
0.64
1
5
2
1
3.34
13
1
15.19
0.30
2
2
2
10.47
25
1
2.07
10.48
26
2
28
1.34
7
11
0.56
3
2
2
2
0.29
One
1
1
Sub. #
U
T
S
Q
P
O
M
L
K
J
I
H
G
F
E
D
C
Letter
20
7
21
19
29
22
27
13
9
10
5
28
11
15
1
26
25
8
4
24
18
17
23
14
12
30
3
16
6
2
Stage One
6.08
1.18
6.72
5.47
15.90
7.83
11.82
3.65
1.45
1.77
0.65
12.82
1.88
3.81
0.27
11.04
10.97
1.42
0.59
10.52
5.16
4.62
10.08
3.73
2.48
16.88
0.54
3.90
0.84
0.29
Ten
1
30
1
1
1
27
29
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
1
1
—
13.27
—
—
—
2.71
4.46
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
3.65
—
—
One
1
30
1
1
1
28
29
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
27
1
1
Stage Two
—
—
—
18.84
—
—
—
1.74
1.84
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
1.69
Ten
19
26
18
16
28
23
29
13
8
9
5
27
10
12
2
24
25
7
3
22
17
14
21
15
11
30
4
20
6
1
6.22
14.72
5.96
4.18
15.37
10.42
15.90
3.43
1.38
1.63
0.64
15.19
2.07
3.34
0.30
10.47
10.48
1.34
0.56
10.36
4.26
3.65
9.52
3.78
2.52
16.71
0.57
7.52
0.87
0.29
One
19
30
20
17
28
21
27
12
8
9
5
26
10
14
1
25
24
7
4
23
16
15
22
13
11
29
3
18
6
2
Total
6.08
20.02
6.72
5.47
15.90
9.57
13.65
3.65
1.45
1.77
0.65
12.82
1.88
3.81
0.28
11.04
10.97
1.42
0.59
10.52
5.16
4.62
10.08
3.73
2.48
16.88
0.54
5.59
0.84
0.29
Ten
0.0226
0.0571
NA
0.0685
0.1218
0.0253
7
30
9
21
4
0.0222
3
0.1272
2
0.1308
23
22
0.0766
0.0818
15
13
0.2921
0.2995
28
0.0351
6
0.0625
8
29
0.0875
0.0883
17
16
0.0712
0.0786
10
0.0257
5
14
0.1576
26
0.1607
0.1089
19
27
0.1082
0.1111
18
0.0723
0.0745
12
20
0.0197
1
11
0.1337
0.1335
25
24
FNIR @ FPIR = 10−3
Table 37: Tabulation of mean identification time results for Class A — Left Index — less than 20-second searches. Letter refers to the participant’s letter code found on the footer of this page.
Sub. # is an identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total
indicating the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are reported in seconds, but
were originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. The number to the left of a value provides the value’s
column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 7 are reprinted to the right of this table.
< 20 seconds
Participant
FPVTE – F INGERPRINT M ATCHING
129
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
130
6
—
24.39
1
1
6
—
—
—
3.25
1
4
3
5
2
66.23
25.66
45.59
43.24
54.24
42.64
6
2
4
3
5
1
66.39
40.14
46.37
44.66
54.61
21.75
2
6
4
3
5
1
0.0252
0.1178
0.0650
0.0278
0.1086
0.0197
FNIR @ FPIR = 10−3
18.50
1
—
1
0.40
6
Total
1
54.61
1
—
5
—
Stage Two
18.26
5
44.66
1
0.46
1
Stage One
Sub. #
1
54.24
3
46.37
5
—
Participant
Letter
2
5
43.24
4
39.74
1
Ten
D
2
3
45.59
2
66.39
One
G
2
4
25.20
6
Ten
I
2
2
66.23
One
S
2
6
Ten
U
2
One
V
Table 38: Tabulation of mean identification time results for Class A — Left Index — greater than or equal to 20-second searches. Letter refers to the participant’s letter code found on the footer
of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in
Section 5, with Total indicating the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are
reported in seconds, but were originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. The number to the left of a
value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 7 are reprinted to the right of this
table.
≥ 20 seconds
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
0.76
3.72
0.45
15.89
2.52
3.71
10.12
3.58
4.13
10.19
6
17
3
30
12
16
23
15
19
24
1
1
2
1
1
1
1
1
1.48
1.38
3.49
11.10
7.43
15.08
3.82
5.64
1.30
5.95
10
9
14
27
22
29
18
20
8
21
2
1
2
1
1
1
1
1
V
2
2
0.59
1
5
2
1
3.26
13
1
11.31
0.29
2
2
2
10.30
25
1
1.75
10.42
26
2
28
1.25
7
11
0.53
4
2
2
2
0.25
One
1
1
Sub. #
U
T
S
Q
P
O
M
L
K
J
I
H
G
F
E
D
C
Letter
19
7
21
20
29
22
28
13
9
10
5
27
11
16
2
26
25
8
4
23
18
17
24
14
12
30
3
15
6
1
Stage One
5.88
1.06
6.83
6.12
15.12
7.66
11.54
3.48
1.37
1.67
0.62
10.80
1.71
3.74
0.27
10.73
10.65
1.33
0.55
10.06
5.13
4.61
10.32
3.52
2.37
16.40
0.51
3.72
0.77
0.27
Ten
1
30
1
1
1
27
28
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
29
1
1
—
10.72
—
—
—
2.71
3.34
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
3.96
—
—
One
1
30
1
1
1
28
29
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
27
1
1
Stage Two
—
—
—
16.39
—
—
—
1.72
1.76
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
1.63
Ten
19
27
18
16
29
22
28
13
8
9
5
26
10
12
2
24
25
7
4
23
17
14
21
15
11
30
3
20
6
1
5.95
12.02
5.64
3.82
15.08
10.15
14.44
3.49
1.38
1.48
0.59
11.31
1.75
3.26
0.29
10.30
10.42
1.25
0.53
10.19
4.13
3.58
10.12
3.71
2.52
15.89
0.45
7.68
0.76
0.25
One
18
30
20
19
28
21
27
12
8
9
5
26
10
14
2
25
24
7
4
22
16
15
23
13
11
29
3
17
6
1
Total
5.88
17.45
6.83
6.12
15.12
9.37
13.30
3.48
1.37
1.67
0.62
10.80
1.71
3.74
0.28
10.73
10.65
1.33
0.55
10.06
5.13
4.61
10.32
3.52
2.37
16.40
0.51
5.36
0.77
0.27
Ten
0.0996
0.0223
21
5
0.1929
28
0.0562
0.0442
7
9
0.0214
0.0218
2
0.1100
4
0.1133
25
22
0.0675
0.0776
17
13
0.2526
0.2615
29
0.0295
6
0.0505
8
30
0.0685
0.0682
14
15
0.0643
0.0708
12
0.0215
3
16
0.1230
26
0.1249
0.0910
19
27
0.0903
0.0933
18
0.0624
0.0630
11
20
0.0190
1
10
0.1124
0.1132
23
24
FNIR @ FPIR = 10−3
Table 39: Tabulation of mean identification time results for Class A — Right Index — less than 20-second searches. Letter refers to the participant’s letter code found on the footer of this page.
Sub. # is an identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total
indicating the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are reported in seconds, but
were originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. The number to the left of a value provides the value’s
column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 8 are reprinted to the right of this table.
< 20 seconds
Participant
FPVTE – F INGERPRINT M ATCHING
131
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
132
6
—
11.90
1
1
6
—
—
—
3.15
1
4
3
5
2
64.47
22.39
43.56
41.60
57.88
29.26
6
2
4
3
5
1
63.54
35.39
44.05
43.20
56.87
20.89
3
6
4
2
5
1
0.0222
0.1007
0.0503
0.0214
0.0909
0.0190
FNIR @ FPIR = 10−3
17.75
1
—
1
0.39
6
Total
1
56.87
1
—
5
—
Stage Two
17.36
5
43.20
1
0.42
1
Stage One
Sub. #
1
57.88
3
44.05
5
—
Participant
Letter
2
5
41.60
4
35.01
1
Ten
D
2
3
43.56
2
63.54
One
G
2
4
21.97
6
Ten
I
2
2
64.47
One
S
2
6
Ten
U
2
One
V
Table 40: Tabulation of mean identification time results for Class A — Right Index — greater than or equal to 20-second searches. Letter refers to the participant’s letter code found on the
footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined
in Section 5, with Total indicating the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are
reported in seconds, but were originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. The number to the left of a
value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 8 are reprinted to the right of this
table.
≥ 20 seconds
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
< 20 seconds
1
1
1
1
1
1
I
J
K
L
O
V
2.39
6.34
6.27
18.92
14.00
18.32
2.20
15.34
9.50
2
4
3
9
6
8
1
7
5
One
5
7
2
8
6
9
4
3
1
Stage One
9.59
16.58
5.03
18.82
15.16
20.61
7.48
6.55
2.66
Ten
1
1
1
1
1
1
1
1
1
—
—
—
—
—
—
—
—
—
One
1
1
1
1
1
1
1
1
1
Stage Two
—
—
—
—
—
—
—
—
—
Ten
5
7
1
8
6
9
3
4
2
9.50
15.34
2.20
18.32
14.00
18.92
6.27
6.34
2.39
One
5
7
2
8
6
9
4
3
1
Total
9.59
16.58
5.03
18.82
15.16
20.61
7.48
6.55
2.66
Ten
0.0374
0.0515
0.0058
0.0143
0.0360
0.0146
0.0229
0.0034
9
2
3
6
4
5
1
0.0368
8
7
FNIR @ FPIR = 10−3
Table 41: Tabulation of mean identification time results for Class A — Left and Right Index — less than 20-second searches. Letter refers to the participant’s letter code found on the footer of
this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5,
with Total indicating the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are reported in
seconds, but were originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. The number to the left of a value provides
the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 9 are reprinted to the right of this table.
1
2
1
Sub. #
G
C
Letter
Participant
FPVTE – F INGERPRINT M ATCHING
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
133
FPVTE – F INGERPRINT M ATCHING
134
16
234.52
73.01
27
2
21
5
23.15
190.00
37.95
10
11
4
4
0.0202
0.0207
0.0030
0.0030
FNIR @ FPIR = 10−3
23
22.27
Ten
18.74
2
Total
23
19.00
27
One
53.93
24
—
Ten
24
70.74
1
Stage Two
19.21
26
1
One
1
170.99
—
Ten
19.08
21
1
Stage One
1
163.78
1
One
1
21
23.15
Sub. #
2
3
0.0412
27
22
0.0311
22.27
15
0.0686
3
69.19
24
0.0684
27
13
244.70
23
0.0030
1
22
40.35
4
H
M
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
Participant
Letter
D
E
71.45
7
55.70
0.0386
15
227.27
10
21
22
37.65
24
57.53
—
8
53.77
0.0143
512.38
1
—
12
8
11
1
—
25
38.86
405.27
0.0286
59.30
—
1
—
6
14
0.0072
500.30
1
—
1
36.19
385.14
7
NA
13
1
—
1
7
32.99
26
NA
—
69.19
1
—
—
4
70.30
25
0.0214
—
13
244.70
1
—
32.68
14
31.32
12
1
23
40.35
1
1
6
23.54
3
174.39
—
71.45
8
55.70
—
3
32.59
8
19
—
16
227.27
10
—
—
5
183.20
1
1
23
37.65
24
1
1
—
20
57.53
2
10
53.77
38.86
405.27
—
1
—
10
512.38
2
13
7
1
—
1
—
11
1
25
36.19
385.14
32.99
1
—
1
59.30
2
9
6
70.30
1
1
500.30
2
32.68
14
31.32
—
14
2
8
23.53
5
1
2
J
I
2
4
32.59
1
F
K
2
7
174.39
G
L
1
9
22
0.0333
183.20
16
0.0027
22
1
0.0027
12
115.02
1
0.0281
2
16
247.48
13
0.0195
0.0370
23
180.13
9
20
114.37
20
19.75
50.30
17
213.08
1
62.27
21
163.65
26
12
—
19
20.11
45.52
1
81.88
1
63.41
26
85.04
26
14
—
27
—
—
1
59.45
1
—
25
75.04
1
1
115.02
27
—
—
17
165.60
1
—
20
95.09
1
1
114.37
16
19.75
50.30
18
153.63
2
62.27
1
20
88.60
26
12
2
17
20.11
45.52
1
2
NA
63.40
2
26
27
15
1
88.81
0.0366
432.22
19
0.0336
15
126.53
17
0.0358
26.96
17
55.33
18
0.0028
429.02
38.91
9
412.64
3
4
9
47.60
25
131.53
—
—
11
252.04
18
—
1
26.86
24
133.45
1
—
25
0.72
18
—
1
18.77
22
—
—
126.53
23
0.79
1
1
18
28.46
22
—
88.81
38.91
4
411.92
1
432.22
1
11
28.83
25
131.53
15
2
6
251.26
19
26.96
1
24
133.45
429.02
2
19
5
2
2
2
O
P
Q
S
T
U
V
Table 42: Tabulation of mean identification time results for Class A — Left and Right Index — greater than or equal to 20-second searches. Letter refers to the participant’s letter code found on
the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined
in Section 5, with Total indicating the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are
reported in seconds, but were originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. The number to the left of a
value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 9 are reprinted to the right of this
table.
≥ 20 seconds
47.51
78.95
79.01
33.07
42.62
36.28
48.01
16
27
28
10
14
11
18
2
1
2
1
2
1
2
47.81
17
1
1
78.80
2
26
80.54
29
1
2
26.88
7
2
38.17
51.59
21
1
12
63.06
25
2
1
52.05
22
1
90.51
49.13
20
2
30
62.98
24
1
2
19.65
6
2
32.39
46.18
15
1
9
31.92
8
2
1
41.57
13
1
5.51
3.71
1
12.52
55.36
23
2
5
48.97
19
1
2
6.62
4
2
3
3.71
One
2
1
Sub. #
NA
—
14
50.63
36.96
NA
—
10
79.17
79.27
53.38
54.11
83.65
41.76
95.91
34.52
30.67
10.07
85.49
28.69
60.52
71.35
60.24
61.06
64.89
20.00
48.76
33.70
41.51
4.13
84.23
72.20
7.00
3.89
Ten
23
24
15
16
25
12
28
9
7
4
27
6
18
21
17
19
20
5
13
8
11
2
26
22
3
1
Stage One
1
1
29
30
1
1
26
25
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
27
1
1
—
—
47.88
48.79
—
—
3.54
3.52
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
4.29
4.12
—
—
One
1
1
—
—
1
1
26
25
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
27
1
1
Stage Two
—
—
—
—
NA
NA
—
—
3.35
3.24
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
9.21
6.82
Ten
14
10
29
28
26
25
16
17
24
11
30
9
5
3
27
7
18
23
19
15
22
6
13
8
12
1
21
20
4
2
48.01
36.28
90.50
81.85
79.01
78.95
51.05
51.33
78.80
38.17
90.51
32.39
12.52
5.51
80.54
26.88
51.59
63.06
52.05
49.13
62.98
19.65
46.18
31.92
41.57
3.71
59.65
53.09
6.62
3.71
One
14
10
—
—
23
24
15
16
25
12
28
9
7
4
26
6
18
21
17
19
20
5
13
8
11
2
27
22
3
1
Total
50.63
36.96
NA
NA
79.17
79.27
56.74
57.34
83.65
41.76
95.91
34.52
30.67
10.08
85.49
28.69
60.52
71.35
60.24
61.06
64.89
20.00
48.76
33.70
41.51
4.13
93.44
79.02
7.00
3.89
Ten
0.0654
0.0647
0.0163
0.0142
0.0259
0.0187
0.1684
0.1681
0.0371
0.0325
0.0998
0.1008
0.0116
0.0094
0.0287
0.0236
0.0288
0.0276
0.1736
0.1634
0.0257
0.0254
0.0098
0.0099
0.1089
0.1133
0.0500
0.0461
0.0192
0.0190
22
21
6
5
13
7
29
28
18
17
23
24
4
1
15
10
16
14
30
27
12
11
2
3
25
26
20
19
9
8
FNIR @ FPIR = 10−3
Table 43: Tabulation of mean identification time results for Class B — Left Slap. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to
differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating the combined
search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are reported in seconds, but were originally recorded
to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the value could not be
performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from
Table 10 are reprinted to the right of this table.
V
U
S
Q
O
M
L
J
I
H
G
F
E
D
C
Letter
Participant
FPVTE – F INGERPRINT M ATCHING
135
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
136
—
4
1
53.78
6.54
3.70
24
3
1
79.03
6.74
3.74
13
2
6
23
24
0.0151
0.0052
0.0072
0.0392
0.0403
FNIR @ FPIR = 10−3
1
—
19
94.98
Total
—
1
6.57
2
27
Stage Two
1
—
27
58.67
Stage One
3.74
1
3.71
2
20
Ten
1
6.74
27
9.49
One
3.70
3
72.46
1
28
Ten
1
6.54
22
4.69
One
1
4
50.07
1
28
Ten
2
19
85.48
One
1
2
27
0.1222
53.98
29
0.1220
2
28
0.0212
22
34.19
18
0.0198
2
8
49.37
16
0.0641
0.0083
14
12.33
25
7
32.28
5
44.17
4.14
8
46.58
13
40.45
14
11.80
18
11
—
5
41.12
4.68
1
—
13
40.27
1
—
17
12
—
1
—
—
1
—
1
—
1
—
1
1
34.19
1
—
—
8
49.37
1
—
14
12.33
1
1
32.28
5
44.17
4.14
8
46.58
13
40.45
2
16
11.80
18
11
1
5
41.12
4.68
2
14
0.0647
40.27
1
20
26
13
2
61.22
0.0058
60.09
5
0.0045
16
71.23
1
0.0156
50.97
21
60.26
14
0.0126
52.86
63.57
17
28.48
10
18
23
49.18
6
84.26
—
—
16
25.86
26
—
1
—
7
77.67
1
—
1
—
27
—
1
—
1
—
—
71.23
1
—
1
0.0167
1
21
60.26
1
—
15
61.22
63.57
17
28.48
1
4
60.09
2
25
49.18
6
84.26
3
16
1
18
25.86
26
1
50.97
2
7
77.67
1
0.0202
52.86
1
29
4
17
21
2
3
10.59
0.1259
31.95
30
0.1155
7
35.23
27
0.0142
5.49
9
96.58
12
0.0132
12.66
33.43
28
41.40
11
6
9
91.42
12
82.92
—
—
30
38.08
25
—
1
—
11
77.37
1
—
1
—
26
—
1
—
1
—
—
35.23
1
—
1
0.0057
1
9
96.58
1
—
3
10.59
33.43
28
41.40
1
20
31.95
2
10
91.42
12
82.92
22
7
1
30
38.08
25
25
5.49
2
12
77.37
26
12.66
1
28
20
6
2
24
0.0369
0.0057
21
0.0381
3
22
0.0266
68.15
76.78
19
0.0273
66.31
22
77.18
20
0.0106
19
74.29
23
NA
8
0.0110
62.95
25
73.96
—
NA
9
61.29
—
24
89.71
—
37.19
21
1
—
29
88.88
10
50.37
3.29
—
1
NA
28
35.66
15
3.35
1
—
—
NA
10
48.47
26
76.78
1
56.35
—
—
15
3.67
23
77.18
30
46.01
1
—
3.56
74.29
24
NA
29
—
1
25
2
27
73.96
—
NA
1
—
64.86
1
26
33.35
—
37.19
1
62.96
2
9
42.87
10
50.37
19
1
15
35.66
15
59.28
2
11
48.47
57.74
1
17
23
2
1
1
1
1
Sub. #
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Table 44: Tabulation of mean identification time results for Class B — Right Slap. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to
differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating the combined
search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are reported in seconds, but were originally recorded
to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the value could not be
performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from
Table 11 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
52.85
85.05
84.02
30.21
40.05
39.24
52.97
19
30
29
10
14
12
20
2
1
2
1
2
1
2
62.90
22
1
1
74.57
2
27
75.92
28
1
2
27.52
7
2
40.42
57.94
21
1
15
47.35
17
2
1
72.35
26
1
49.50
70.12
24
2
18
42.86
16
1
2
5.52
1
2
28.89
38.37
11
1
9
28.72
8
2
1
39.56
13
1
20.85
8.18
3
10.37
71.41
25
2
6
67.64
23
1
2
10.63
5
2
4
7.36
One
2
1
Sub. #
NA
—
16
55.45
41.22
NA
—
11
87.27
87.49
61.10
71.42
78.24
43.46
52.63
30.58
63.37
22.54
80.17
29.91
73.10
53.29
84.76
85.94
43.96
5.43
40.60
30.42
40.38
7.40
124.70
120.11
11.08
7.86
Ten
25
26
17
19
21
12
14
8
18
5
22
6
20
15
23
24
13
1
10
7
9
2
28
27
4
3
Stage One
1
1
29
30
1
1
25
26
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
27
1
1
—
—
50.99
56.68
—
—
1.03
1.10
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
2.80
1.72
—
—
One
1
1
—
—
1
1
25
26
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
27
1
1
Stage Two
—
—
—
—
NA
NA
—
—
0.78
0.84
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
5.36
3.14
Ten
17
11
30
29
27
28
18
20
25
13
16
9
6
4
26
7
19
15
23
22
14
1
10
8
12
3
24
21
5
2
52.97
39.24
91.04
86.88
84.02
85.05
53.88
63.99
74.57
40.42
49.50
28.89
20.85
10.37
75.92
27.52
57.94
47.35
72.35
70.12
42.86
5.52
38.37
28.72
39.56
8.18
74.21
69.35
10.63
7.36
One
16
11
—
—
25
26
17
19
21
12
14
8
18
5
22
6
20
15
23
24
13
1
10
7
9
2
28
27
4
3
Total
55.45
41.22
NA
NA
87.27
87.49
61.88
72.27
78.24
43.46
52.63
30.58
63.37
22.54
80.17
29.91
73.10
53.29
84.76
85.94
43.96
5.43
40.60
30.42
40.38
7.40
130.06
123.25
11.08
7.86
Ten
NA
NA
0.0031
0.0024
0.0063
0.0049
0.0910
0.0901
0.0106
0.0084
0.0349
0.0361
0.0022
0.0015
0.0068
0.0047
0.0054
0.0062
0.0904
0.0882
0.0057
0.0051
0.0021
0.0022
0.0160
0.0190
0.0139
0.0124
0.0036
0.0036
30
29
6
5
15
10
28
26
18
17
23
24
3
1
16
9
12
14
27
25
13
11
2
3
21
22
20
19
7
7
FNIR @ FPIR = 10−3
Table 45: Tabulation of mean identification time results for Class B — Left and Right Slap. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used
to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating the combined
search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are reported in seconds, but were originally recorded
to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the value could not be
performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from
Table 12 are reprinted to the right of this table.
V
U
S
Q
O
M
L
J
I
H
G
F
E
D
C
Letter
Participant
FPVTE – F INGERPRINT M ATCHING
137
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
138
—
4
3
52.42
10.28
9.00
26
4
3
98.05
10.71
9.11
16
2
6
29
30
0.0043
0.0012
0.0020
NA
NA
FNIR @ FPIR = 10−3
1
—
17
105.02
Total
—
1
1.80
2
28
Stage Two
1
—
27
54.37
Stage One
9.11
1
1.00
2
19
Ten
3
10.71
25
3.26
One
9.00
4
96.25
1
28
Ten
3
10.28
25
1.80
One
1
4
51.42
1
28
Ten
2
18
101.77
One
1
2
28
0.0591
52.57
27
0.0591
2
27
0.0062
20
40.86
18
0.0040
2
11
54.45
14
0.0203
0.0024
14
5.99
23
7
39.12
1
27.42
8.63
12
51.59
5
54.84
16
6.33
25
15
—
1
31.67
8.76
1
—
9
52.88
1
—
26
18
—
1
—
—
1
—
1
—
1
—
1
1
40.86
1
—
—
11
54.45
1
—
14
5.99
1
1
39.12
1
27.42
8.63
13
51.59
5
54.84
2
19
6.33
26
15
1
1
31.67
8.76
2
10
0.0204
52.88
1
27
24
21
2
97.35
0.0012
98.29
2
0.0009
27
29.56
1
0.0049
82.43
7
78.97
17
0.0033
85.74
24.57
20
28.07
11
27
6
60.01
6
69.51
—
—
20
26.31
18
—
1
—
7
64.38
1
—
1
—
22
—
1
—
1
—
—
29.56
1
—
1
0.0031
1
7
78.97
1
—
10
97.35
24.57
20
28.07
1
8
98.29
2
6
60.01
6
69.51
5
27
1
22
26.31
18
1
82.43
2
7
64.38
1
0.0033
85.74
1
24
8
11
28
2
5
34.20
0.0543
86.45
26
0.0515
22
43.38
25
0.0041
14.42
12
74.53
15
0.0035
28.56
41.37
19
39.70
13
8
13
70.45
10
68.54
—
—
23
37.04
17
—
1
—
11
63.87
1
—
1
—
21
—
1
—
1
—
—
43.38
1
—
1
0.0012
1
12
74.53
1
—
2
34.20
41.37
19
39.70
1
16
86.45
2
14
70.45
10
68.54
14
22
1
26
37.04
17
25
14.42
2
12
63.87
27
28.56
1
23
16
8
2
16
0.0108
0.0012
20
0.0136
2
21
0.0099
57.50
91.50
19
0.0141
82.97
23
93.61
22
0.0027
21
86.50
24
NA
9
0.0024
48.85
28
88.70
—
NA
7
71.67
—
29
88.83
—
37.75
24
1
—
30
80.14
9
51.37
1.02
—
1
NA
25
35.51
13
1.05
1
—
—
NA
10
49.72
26
91.50
1
58.50
—
—
15
1.47
23
93.61
30
37.63
1
—
1.46
86.50
24
NA
29
—
1
26
2
29
88.70
—
NA
1
—
56.48
1
30
30.33
—
37.75
1
81.92
2
9
42.51
9
51.37
21
1
15
35.50
13
47.38
2
11
49.72
70.21
1
17
25
2
1
1
1
1
Sub. #
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Table 46: Tabulation of mean identification time results for Class B — Identification Flats. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used
to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating the combined
search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are reported in seconds, but were originally recorded
to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the value could not be
performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from
Table 13 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
41.98
87.76
89.24
47.49
50.30
47.76
63.19
8
29
30
13
15
14
19
2
1
2
1
2
1
2
43.10
10
1
1
82.13
2
24
85.17
26
1
2
46.32
12
2
73.31
56.93
17
1
22
79.70
23
2
1
64.90
20
1
45.96
67.90
2
11
37.15
7
21
1
2
11.21
1
2
42.60
60.33
18
1
9
50.99
16
2
1
85.50
27
1
21.26
15.94
2
17.53
82.82
25
2
6
85.52
28
1
2
18.31
4
2
3
21.04
One
5
1
Sub. #
NA
—
16
66.29
49.30
NA
—
11
92.85
97.78
45.57
47.65
91.94
80.40
45.71
42.17
67.26
39.66
94.45
50.85
64.06
84.04
72.52
77.93
48.42
14.65
60.33
50.83
88.32
15.93
166.16
173.53
19.18
22.92
Ten
24
26
7
9
23
20
8
6
17
5
25
13
15
21
18
19
10
1
14
12
22
2
27
28
3
4
Stage One
1
1
29
30
1
1
27
26
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
25
1
1
—
—
17.68
27.65
—
—
1.70
1.64
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
2.09
1.55
—
—
One
1
1
—
—
1
1
25
26
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
28
27
1
1
Stage Two
—
—
—
—
NA
NA
—
—
1.39
1.54
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
2.98
2.52
Ten
17
13
20
22
30
29
9
10
24
21
11
8
6
3
26
12
15
23
18
19
7
1
16
14
27
2
25
28
4
5
63.19
47.76
67.98
75.14
89.24
87.76
43.67
44.75
82.13
73.31
45.96
42.60
21.26
17.53
85.17
46.32
56.93
79.70
64.90
67.91
37.15
11.21
60.34
50.99
85.50
15.94
84.92
87.07
18.31
21.04
One
16
11
—
—
24
26
8
10
23
20
7
6
17
5
25
13
15
21
18
19
9
1
14
12
22
2
27
28
3
4
Total
66.29
49.30
NA
NA
92.85
97.78
46.96
49.18
91.94
80.40
45.71
42.17
67.27
39.67
94.45
50.85
64.06
84.04
72.52
77.93
48.42
14.65
60.33
50.83
88.32
15.93
169.13
176.06
19.18
22.92
Ten
NA
0.0711
0.0015
0.0011
0.0088
0.0048
0.0734
0.0734
0.0368
0.0276
0.0276
0.0275
0.0013
0.0010
0.0047
0.0027
0.0102
0.0095
0.0934
0.0826
0.0025
0.0027
0.0011
0.0013
0.0311
0.1680
0.0163
0.0155
0.0024
0.0024
30
24
6
2
14
13
25
25
23
20
20
19
4
1
12
10
16
15
28
27
9
10
2
4
22
29
18
17
7
7
FNIR @ FPIR = 10−3
Table 47: Tabulation of mean identification time results for Class C — Ten-Finger Plain-to-Plain. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an
identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating
the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are reported in seconds, but were
originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the
value could not be performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference,
the FNIR values from Table 14 are reprinted to the right of this table.
V
U
S
Q
O
M
L
J
I
H
G
F
E
D
C
Letter
Participant
FPVTE – F INGERPRINT M ATCHING
139
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
140
—
5
4
65.97
25.91
25.25
27
4
3
104.36
27.89
27.50
18
7
4
15
16
0.0106
0.0018
0.0015
0.0085
0.0094
FNIR @ FPIR = 10−3
1
—
17
163.56
Total
—
1
1.91
1
28
Stage Two
1
—
25
86.39
Stage One
27.50
1
4.17
1
30
Ten
3
27.89
28
3.02
One
25.25
4
102.45
1
28
Ten
4
25.91
27
2.81
One
1
5
61.79
1
27
Ten
2
18
160.54
One
1
1
28
0.0536
83.58
25
0.0536
1
25
0.0447
28
58.39
24
0.0333
2
11
71.58
21
0.0201
0.0050
18
12.99
20
12
55.63
2
31.33
59.56
13
68.61
5
10.54
18
11.67
25
13
—
2
30.61
9.52
1
—
6
57.02
1
—
28
14
—
1
—
—
1
—
1
—
1
—
1
1
58.39
1
—
—
11
71.58
1
—
18
12.99
1
1
55.63
2
31.33
59.56
15
68.61
5
10.54
2
20
11.67
25
13
1
2
30.61
9.52
2
6
0.0199
57.01
1
29
19
16
2
94.22
0.0013
94.72
1
0.0014
26
75.61
2
0.0051
84.14
19
41.07
13
0.0033
84.26
79.04
7
40.14
9
29
24
40.53
6
81.37
—
—
9
36.09
22
—
1
—
8
72.62
1
—
1
—
20
—
1
—
1
—
—
75.61
1
—
1
0.0097
1
19
41.07
1
—
17
94.22
79.04
7
40.14
1
14
94.72
2
25
40.53
6
81.37
7
26
1
9
36.09
23
1
84.14
2
8
72.62
1
0.0083
84.26
1
22
14
14
30
2
7
64.19
0.0783
58.45
28
0.0716
12
49.47
27
0.0034
31.42
9
50.67
11
0.0033
19.52
47.38
10
66.17
9
3
11
48.71
15
81.31
—
—
12
59.01
21
—
1
—
15
72.55
1
—
1
—
19
—
1
—
1
—
—
49.47
1
—
1
0.0017
1
9
50.67
1
—
5
64.18
47.38
10
66.17
1
24
58.45
2
11
48.71
15
81.31
26
12
1
13
59.01
22
26
31.42
2
17
72.55
25
19.51
1
21
24
3
2
26
0.0860
0.0014
29
0.2462
2
30
0.0358
85.39
80.86
23
0.0351
83.58
20
71.01
22
0.0017
23
74.01
17
NA
5
0.0019
83.25
22
72.95
—
NA
8
83.35
—
21
82.76
—
42.49
27
1
—
25
74.39
8
67.92
2.53
—
1
NA
23
40.94
16
2.55
1
—
—
NA
10
65.47
27
80.86
1
34.24
—
—
16
2.23
20
71.01
30
21.36
1
—
2.30
74.01
17
NA
29
—
1
26
2
24
72.95
—
NA
1
—
82.87
1
23
48.52
—
42.49
1
81.03
2
12
53.02
8
67.92
21
1
14
40.94
16
81.02
2
10
65.47
81.05
1
19
27
2
1
1
1
1
Sub. #
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Table 48: Tabulation of mean identification time results for Class C — Ten-Finger Rolled-to-Rolled. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an
identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating
the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are reported in seconds, but were
originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the
value could not be performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference,
the FNIR values from Table 15 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
47.87
89.37
89.34
48.44
53.32
48.91
71.28
10
30
29
11
13
12
24
2
1
2
1
2
1
2
47.17
9
1
1
75.98
2
26
79.37
28
1
2
42.13
8
2
65.71
66.90
23
1
22
76.98
27
2
1
65.69
21
1
57.90
65.03
2
16
34.47
6
20
1
2
25.05
3
2
57.06
61.82
19
1
15
58.39
17
2
1
59.20
18
1
18.62
10.81
1
27.43
74.38
25
2
2
55.60
14
1
2
33.93
5
2
4
36.55
One
7
1
Sub. #
NA
—
19
74.86
50.94
NA
—
8
90.28
96.29
51.32
52.18
86.80
74.07
58.12
56.59
50.64
58.35
89.84
46.98
76.96
82.78
73.62
78.10
45.09
32.02
61.34
58.89
61.17
11.40
155.17
98.54
35.83
40.68
Ten
25
26
9
10
23
18
12
11
7
13
24
6
20
22
17
21
5
2
16
14
15
1
28
27
3
4
Stage One
1
1
29
30
1
1
27
28
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
26
25
1
1
—
—
20.14
34.07
—
—
1.85
1.91
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
1.62
0.77
—
—
One
1
1
—
—
1
1
28
26
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
27
25
1
1
Stage Two
—
—
—
—
NA
NA
—
—
2.36
1.72
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
1.90
0.71
Ten
22
9
23
28
29
30
11
10
24
20
14
13
2
4
27
8
21
26
19
18
6
3
17
15
16
1
25
12
5
7
71.28
48.91
73.46
82.50
89.34
89.37
49.72
49.07
75.98
65.71
57.90
57.06
18.62
27.43
79.37
42.13
66.90
76.98
65.69
65.03
34.47
25.05
61.82
58.39
59.20
10.81
76.00
56.38
33.93
36.55
One
19
8
—
—
25
26
9
10
23
18
12
11
7
13
24
6
20
22
17
21
5
2
16
14
15
1
28
27
3
4
Total
74.86
50.94
NA
NA
90.28
96.29
53.68
53.90
86.80
74.07
58.12
56.59
50.65
58.36
89.84
46.98
76.96
82.78
73.62
78.10
45.09
32.02
61.34
58.89
61.17
11.40
157.06
99.25
35.83
40.68
Ten
0.0149
0.0169
0.0028
0.0018
0.0137
0.0056
0.2514
0.2514
0.0649
0.0521
0.0291
0.0285
0.0014
0.0011
0.0071
0.0034
0.0136
0.0129
0.3067
0.2514
0.0041
0.0036
0.0020
0.0022
0.1017
0.2366
0.0378
0.0295
0.0052
0.0039
17
18
6
3
16
12
27
27
24
23
20
19
2
1
13
7
15
14
30
27
10
8
4
5
25
26
22
21
11
9
FNIR @ FPIR = 10−3
Table 49: Tabulation of mean identification time results for Class C — Ten-Finger Plain-to-Rolled. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an
identifier used to differentiate between the two submissions each participant could make. Stage One and Stage Two refer to the stages of identification defined in Section 5, with Total indicating
the combined search time. One and Ten refer to the number of concurrent identification processes run on a compute node. All values are mean durations are reported in seconds, but were
originally recorded to microsecond precision. A — indicates that the operation completed faster than could be reliably measured. NA indicates that the operations required to produce the
value could not be performed. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference,
the FNIR values from Table 16 are reprinted to the right of this table.
V
U
S
Q
O
M
L
J
I
H
G
F
E
D
C
Letter
Participant
FPVTE – F INGERPRINT M ATCHING
141
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
142
E
FPVTE – F INGERPRINT M ATCHING
Progression for Last Two Submissions
This appendix provides additional information in reference to comments from Section 8 that looked at accuracy versus
search time. The tables in this appendix show search times for the last two of the three submission periods during the
evaluation. Generally, participants attempt to improve accuracy with potential tradeoffs in speed and template size. The
tables in this section show the search times and FNIR @ FPIR = 10−3 for the these two submissions.
Class A results are in Tables 50 through 52, Class B results are in Tables 53 through 56, and Class C results are in Tables 57
through 59.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
Third
FNIR
1
1
0.12
16
0.1077
2
6
0.75
15
1
9
1.79
2
Time
FNIR
1
0.29
24
0.1335
0.0984
6
0.87
25
0.1337
0.0376
20
7.52
1
0.0197
3
0.42
10
0.0794
4
0.57
12
0.0745
2
22
15.40
8
0.0757
30
16.71
11
0.0723
1
11
2.50
18
0.1122
11
2.52
20
0.1111
2
15
3.64
17
0.1093
15
3.78
18
0.1082
1
16
5.30
23
0.1221
21
9.52
19
0.1089
1
5
0.71
20
0.1168
14
3.65
26
0.1576
2
7
0.94
19
0.1160
17
4.26
27
0.1607
1
18
6.19
1
0.0306
22
10.36
5
0.0257
1
2
0.35
13
0.0900
3
0.56
14
0.0786
2
4
0.60
9
0.0773
7
1.34
10
0.0712
1
—
—
—
—
25
10.48
17
0.0883
2
—
—
—
—
24
10.47
16
0.0875
1
8
1.76
14
0.0913
2
0.30
8
0.0625
2
14
3.04
12
0.0881
12
3.34
6
0.0351
1
—
—
—
—
10
2.07
29
0.2995
2
—
—
—
—
27
15.19
28
0.2921
1
10
1.96
7
0.0751
5
0.64
15
0.0818
2
13
2.95
6
0.0735
9
1.63
13
0.0766
1
—
—
—
—
8
1.38
23
0.1308
2
12
2.83
24
0.1343
13
3.43
22
0.1272
1
20
6.97
4
0.0507
29
15.90
2
0.0222
2
19
6.42
5
0.0511
23
10.42
3
0.0226
1
17
5.99
11
0.0811
28
15.37
7
0.0571
1
23
16.49
21
0.1181
16
4.18
30
NA
2
—
—
—
—
18
5.96
9
0.0685
U
1
24
18.62
22
0.1209
26
14.72
21
0.1218
V
1
21
10.34
3
0.0402
19
6.22
4
0.0253
D
2
1
3.99
1
0.0269
2
42.64
1
0.0197
G
2
4
27.35
5
0.1210
5
54.24
5
0.1086
I
2
3
9.04
2
0.0274
3
43.24
3
0.0278
F
G
H
I
< 20 seconds
Second
Time
Sub. #
1
E
J
K
L
M
O
P
Q
S
T
≥ 20 seconds
143
S
2
2
5.94
4
0.0811
4
45.59
4
0.0650
U
2
—
—
—
—
1
25.66
6
0.1178
V
2
5
61.50
3
0.0395
6
66.23
2
0.0252
Table 50: Tabulation of the progression of identification timing and accuracy for Class A — Left Index. Submissions were split into two groups. The first
group includes submissions that, in Third, performed searches on average in less than 20 seconds, and the second includes those that took, on average,
20 seconds or longer. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the
two submissions each participant could make. The Time column shows the time used to perform a search over an enrollment set of 100 000 in seconds,
but was originally recorded to microsecond precision. The FNIR column shows FNIR for each submission at FPIR = 10−3 . NA indicates that the
operations required to produce the value could not be performed. — indicates that there was not a validated submission during that submission period.
The number to the left of a value provides the value’s column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
144
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
< 20 seconds
I
J
K
L
M
O
Third
FNIR
1
1
0.11
20
0.1022
2
6
0.67
15
1
9
1.77
5
Time
FNIR
1
0.25
24
0.1132
0.0899
6
0.76
23
0.1124
0.0337
20
7.68
1
0.0190
1
2
0.37
12
0.0733
3
0.45
11
0.0630
2
23
14.66
10
0.0709
30
15.89
10
0.0624
1
11
2.26
18
0.0945
11
2.52
20
0.0933
2
15
3.30
17
0.0928
15
3.71
18
0.0903
1
16
5.39
19
0.1018
21
10.12
19
0.0910
1
5
0.65
22
0.1035
14
3.58
26
0.1230
2
7
0.88
21
0.1025
17
4.13
27
0.1249
1
17
5.91
1
0.0232
23
10.19
3
0.0215
1
3
0.37
14
0.0800
4
0.53
16
0.0708
2
4
0.55
9
0.0700
7
1.25
12
0.0643
1
—
—
—
—
25
10.42
14
0.0682
2
—
—
—
—
24
10.30
15
0.0685
1
8
1.54
13
0.0739
2
0.29
8
0.0505
2
14
2.94
11
0.0721
12
3.26
6
0.0295
1
—
—
—
—
10
1.75
30
0.2615
2
—
—
—
—
26
11.31
29
0.2526
1
10
1.89
7
0.0672
5
0.59
17
0.0776
2
13
2.79
8
0.0673
9
1.48
13
0.0675
1
—
—
—
—
8
1.38
25
0.1133
2
12
2.62
24
0.1120
13
3.49
22
0.1100
1
22
12.96
3
0.0248
28
14.44
4
0.0218
2
19
9.23
2
0.0242
22
10.15
2
0.0214
1
18
6.18
6
0.0640
29
15.08
7
0.0442
1
21
12.30
16
0.0924
16
3.82
28
0.1929
2
—
—
—
—
18
5.64
9
0.0562
1
24
15.97
23
0.1048
27
12.02
21
0.0996
V
1
20
9.96
4
0.0331
19
5.95
5
0.0223
D
2
2
6.25
2
0.0248
2
29.26
1
0.0190
G
2
4
26.76
5
0.1007
5
57.88
5
0.0909
I
2
3
9.31
1
0.0217
3
41.60
2
0.0214
S
2
1
6.20
4
0.0641
4
43.56
4
0.0503
U
2
—
—
—
—
1
22.39
6
0.1007
V
2
5
60.50
3
0.0330
6
64.47
3
0.0222
P
Q
S
T
U
≥ 20 seconds
Second
Time
Sub. #
Table 51: Tabulation of the progression of identification timing and accuracy for Class A — Right Index. Submissions were split into two groups. The first
group includes submissions that, in Third, performed searches on average in less than 20 seconds, and the second includes those that took, on average,
20 seconds or longer. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the
two submissions each participant could make. The Time column shows the time used to perform a search over an enrollment set of 100 000 in seconds,
but was originally recorded to microsecond precision. The FNIR column shows FNIR for each submission at FPIR = 10−3 . — indicates that there was
not a validated submission during that submission period. The number to the left of a value provides the value’s column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
< 20 seconds
C
Second
Time
Sub. #
Third
FNIR
Time
FNIR
1
1
1.48
6
0.0313
2
2.39
7
0.0368
2
2
6.30
7
0.0354
4
6.34
8
0.0374
G
1
3
7.38
8
0.1426
3
6.27
9
0.0515
I
1
7
19.73
2
0.0073
9
18.92
2
0.0058
J
1
5
9.20
4
0.0161
6
14.00
3
0.0143
K
1
—
—
—
—
8
18.32
6
0.0360
L
1
6
17.81
5
0.0306
1
2.20
4
0.0146
O
1
8
57.36
3
0.0132
7
15.34
5
0.0229
V
1
4
8.56
1
0.0056
5
9.50
1
0.0034
1
2
14.06
5
0.0046
16
73.01
4
0.0030
2
9
46.57
1
0.0033
23
234.52
4
0.0030
1
4
17.36
10
0.0216
2
22.27
11
0.0207
2
22
463.95
9
0.0208
27
500.30
10
0.0202
1
12
56.49
15
0.0392
13
59.30
21
0.0386
2
13
67.36
16
0.0407
15
71.45
22
0.0412
2
19
166.37
21
0.0558
22
227.27
15
0.0311
1
5
18.42
20
0.0538
8
37.65
24
0.0686
2
6
25.38
19
0.0537
12
53.77
23
0.0684
I
2
18
162.34
4
0.0038
25
385.14
4
0.0030
J
2
3
16.23
8
0.0137
7
36.19
8
0.0143
K
2
—
—
—
—
6
32.68
14
0.0286
L
2
7
34.31
13
0.0365
3
23.54
7
0.0072
1
—
—
—
—
5
32.59
26
NA
2
—
—
—
—
20
183.20
25
NA
2
15
85.37
7
0.0134
10
45.52
12
0.0214
1
—
—
—
—
14
63.41
20
0.0370
2
16
89.78
17
0.0408
17
114.37
16
0.0333
1
14
85.29
3
0.0035
21
213.08
1
0.0027
2
10
48.40
2
0.0034
19
163.65
1
0.0027
1
1
13.37
14
0.0379
1
20.11
13
0.0281
2
11
54.11
11
0.0316
26
429.02
9
0.0195
1
17
138.43
12
0.0318
4
26.96
27
NA
2
—
—
—
—
9
38.91
19
0.0366
D
E
F
G
H
≥ 20 seconds
145
M
O
P
Q
S
T
U
V
1
8
38.87
18
0.0532
11
47.60
17
0.0336
2
21
288.82
22
0.2620
24
252.04
18
0.0358
2
20
197.77
6
0.0048
18
133.45
3
0.0028
Table 52: Tabulation of the progression of identification timing and accuracy for Class A — Left and Right Index. Submissions were split into two groups.
The first group includes submissions that, in Third, performed searches on average in less than 20 seconds, and the second includes those that took, on
average, 20 seconds or longer. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate
between the two submissions each participant could make. The Time column shows the time used to perform a search over an enrollment set of 1 600 000
in seconds, but was originally recorded to microsecond precision. The FNIR column shows FNIR for each submission at FPIR = 10−3 . NA indicates
that the operations required to produce the value could not be performed. — indicates that there was not a validated submission during that submission
period. The number to the left of a value provides the value’s column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
146
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Second
Time
Sub. #
1
1
Third
FNIR
3.20
14
0.0667
Time
2
FNIR
3.71
22
0.0654
2
3
6.18
15
0.0669
4
6.62
21
0.0647
1
14
39.90
5
0.0178
20
53.09
6
0.0163
2
18
55.83
7
0.0242
21
59.65
5
0.0142
1
2
3.37
12
0.0397
1
3.71
13
0.0259
2
7
19.68
8
0.0277
12
41.57
7
0.0187
1
—
—
—
—
8
31.92
29
0.1684
2
—
—
—
—
13
46.18
28
0.1681
1
12
33.42
17
0.0704
6
19.65
18
0.0371
2
19
61.44
16
0.0677
22
62.98
17
0.0325
1
13
39.16
18
0.0798
15
49.13
23
0.0998
2
17
46.21
20
0.0813
19
52.05
24
0.1008
1
21
77.87
4
0.0161
23
63.06
4
0.0116
2
20
67.21
3
0.0127
18
51.59
1
0.0094
1
5
11.15
13
0.0491
7
26.88
15
0.0287
2
11
31.32
9
0.0340
27
80.54
10
0.0236
1
4
10.66
19
0.0809
3
5.51
16
0.0288
2
8
23.68
20
0.0813
5
12.52
14
0.0276
1
—
—
—
—
9
32.39
30
0.1736
2
22
89.64
22
0.1634
30
90.51
27
0.1634
1
6
19.29
11
0.0366
11
38.17
12
0.0257
2
9
30.71
10
0.0349
24
78.80
11
0.0254
1
16
45.67
1
0.0104
17
51.33
2
0.0098
2
10
31.26
2
0.0116
16
51.05
3
0.0099
1
—
—
—
—
25
78.95
25
0.1089
2
—
—
—
—
26
79.01
26
0.1133
1
—
—
—
—
28
81.85
20
0.0500
2
—
—
—
—
29
90.50
19
0.0461
1
15
42.01
6
0.0239
10
36.28
9
0.0192
2
—
—
—
—
14
48.01
8
0.0190
Table 53: Tabulation of the progression of identification timing and accuracy for Class B — Left Slap. Letter refers to the participant’s letter code found
on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. The Time column shows
the time used to perform a search over an enrollment set of 3 000 000 in seconds, but was originally recorded to microsecond precision. The FNIR column
shows FNIR for each submission at FPIR = 10−3 . — indicates that there was not a validated submission during that submission period. The number to
the left of a value provides the value’s column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Second
Time
Sub. #
1
147
1
Third
FNIR
3.16
16
0.0391
Time
1
FNIR
3.70
24
0.0403
2
3
6.10
14
0.0386
4
6.54
23
0.0392
1
13
37.74
5
0.0082
19
53.78
6
0.0072
2
19
56.55
6
0.0100
20
58.67
2
0.0052
1
2
3.71
11
0.0220
2
4.68
13
0.0151
2
9
25.48
7
0.0134
12
40.27
7
0.0083
1
—
—
—
—
8
32.28
29
0.1222
2
—
—
—
—
14
46.58
28
0.1220
1
6
19.93
17
0.0423
5
11.80
18
0.0212
2
15
41.42
15
0.0388
13
41.12
16
0.0198
1
14
39.40
20
0.0554
17
50.97
25
0.0641
2
17
46.02
21
0.0563
18
52.86
26
0.0647
1
21
79.29
4
0.0074
23
63.57
5
0.0058
2
20
77.42
1
0.0057
16
49.18
1
0.0045
1
5
11.62
13
0.0313
7
25.86
14
0.0156
2
11
32.12
9
0.0210
27
77.67
10
0.0126
1
4
11.09
18
0.0526
3
5.49
15
0.0167
2
8
23.42
19
0.0536
6
12.66
17
0.0202
1
—
—
—
—
9
33.43
30
0.1259
2
22
90.28
22
0.1157
30
91.42
27
0.1155
1
7
20.12
12
0.0231
11
38.08
12
0.0142
2
10
32.04
10
0.0212
26
77.37
11
0.0132
1
18
53.98
2
0.0064
22
62.95
3
0.0057
2
12
33.57
3
0.0069
21
61.29
3
0.0057
1
—
—
—
—
25
74.29
21
0.0369
2
—
—
—
—
24
73.96
22
0.0381
1
—
—
—
—
29
89.71
19
0.0266
2
—
—
—
—
28
88.88
20
0.0273
1
16
42.11
8
0.0148
10
35.66
8
0.0106
2
—
—
—
—
15
48.47
9
0.0110
Table 54: Tabulation of the progression of identification timing and accuracy for Class B — Right Slap. Letter refers to the participant’s letter code found
on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. The Time column shows
the time used to perform a search over an enrollment set of 3 000 000 in seconds, but was originally recorded to microsecond precision. The FNIR column
shows FNIR for each submission at FPIR = 10−3 . — indicates that there was not a validated submission during that submission period. The number to
the left of a value provides the value’s column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
148
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Second
Time
Sub. #
1
1
Third
FNIR
3.44
21
0.0365
Time
FNIR
2
7.36
30
NA
2
3
4.33
20
0.0356
5
10.63
29
NA
1
13
38.51
5
0.0040
21
69.35
6
0.0031
2
17
48.86
6
0.0045
24
74.21
5
0.0024
1
4
6.39
11
0.0098
3
8.18
15
0.0063
2
18
52.30
8
0.0074
12
39.56
10
0.0049
1
—
—
—
—
8
28.72
28
0.0910
2
—
—
—
—
10
38.37
26
0.0901
1
2
4.27
13
0.0132
1
5.52
18
0.0106
2
7
22.15
16
0.0235
14
42.86
17
0.0084
1
19
52.34
18
0.0325
22
70.12
23
0.0349
2
20
61.14
19
0.0340
23
72.35
24
0.0361
1
22
79.56
4
0.0031
15
47.35
3
0.0022
2
21
69.84
3
0.0027
19
57.94
1
0.0015
1
5
11.06
15
0.0228
7
27.52
16
0.0068
2
12
32.23
10
0.0090
26
75.92
9
0.0047
1
8
25.03
17
0.0312
4
10.37
12
0.0054
2
10
26.13
14
0.0216
6
20.85
14
0.0062
1
—
—
—
—
9
28.89
27
0.0904
2
16
48.74
22
0.0882
16
49.50
25
0.0882
1
6
19.15
12
0.0106
13
40.42
13
0.0057
2
11
31.55
9
0.0088
25
74.57
11
0.0051
1
15
47.54
1
0.0019
20
63.99
2
0.0021
2
9
25.99
2
0.0020
18
53.88
3
0.0022
1
—
—
—
—
28
85.05
21
0.0160
2
—
—
—
—
27
84.02
22
0.0190
1
—
—
—
—
29
86.88
20
0.0139
2
—
—
—
—
30
91.04
19
0.0124
1
14
46.90
7
0.0051
11
39.24
7
0.0036
2
—
—
—
—
17
52.97
7
0.0036
Table 55: Tabulation of the progression of identification timing and accuracy for Class B — Left and Right Slap. Letter refers to the participant’s letter code
found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. The Time column
shows the time used to perform a search over an enrollment set of 3 000 000 in seconds, but was originally recorded to microsecond precision. The FNIR
column shows FNIR for each submission at FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. —
indicates that there was not a validated submission during that submission period. The number to the left of a value provides the value’s column-wise
ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
149
Second
Time
Sub. #
1
2
2
1
Third
FNIR
4.67
20
0.0276
3
5.01
19
9
26.83
6
2
13
33.98
1
4
7.71
2
20
70.09
1
—
—
2
—
—
1
1
2.50
2
6
1
18
Time
FNIR
3
9.00
30
0.0268
4
10.28
29
NA
0.0028
17
52.42
6
0.0020
5
0.0027
19
54.37
2
0.0012
11
0.0071
2
8.76
16
0.0043
8
0.0043
18
52.88
7
0.0024
—
—
12
39.12
27
0.0591
—
—
16
51.59
27
0.0591
12
0.0101
1
6.33
18
0.0062
12.62
14
0.0157
9
31.67
14
0.0040
56.70
16
0.0220
26
82.43
23
0.0203
2
19
67.68
17
0.0249
27
85.74
24
0.0204
1
22
75.61
4
0.0021
6
24.57
2
0.0012
2
17
50.74
2
0.0012
20
60.01
1
0.0009
1
5
8.13
22
0.0614
7
26.31
17
0.0049
2
11
31.60
9
0.0054
22
64.38
11
0.0033
1
14
36.59
18
0.0250
5
14.42
10
0.0031
2
12
33.50
13
0.0147
8
28.56
11
0.0033
NA
1
—
—
—
—
13
41.37
26
0.0543
2
21
70.37
21
0.0515
23
70.45
25
0.0515
1
7
16.34
15
0.0173
11
37.04
15
0.0041
2
10
30.74
10
0.0056
21
63.87
13
0.0035
1
16
48.59
1
0.0010
14
48.85
2
0.0012
2
8
26.38
2
0.0012
24
71.67
2
0.0012
1
—
—
—
—
28
86.50
20
0.0108
2
—
—
—
—
29
88.70
21
0.0136
1
—
—
—
—
30
88.83
19
0.0099
2
—
—
—
—
25
80.14
22
0.0141
1
15
42.75
7
0.0041
10
35.51
9
0.0027
2
—
—
—
—
15
49.72
7
0.0024
Table 56: Tabulation of the progression of identification timing and accuracy for Class B — Identification Flats. Letter refers to the participant’s letter code
found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. The Time column
shows the time used to perform a search over an enrollment set of 3 000 000 in seconds, but was originally recorded to microsecond precision. The FNIR
column shows FNIR for each submission at FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. —
indicates that there was not a validated submission during that submission period. The number to the left of a value provides the value’s column-wise
ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
150
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Second
Time
Sub. #
Third
FNIR
3
10.27
16
0.0574
2
2
10.04
17
1
10
44.21
4
2
20
74.46
1
6
27.61
2
21
79.95
1
12
50.99
2
17
60.34
1
1
8.43
2
5
1
18
Time
FNIR
5
21.04
30
NA
0.0584
4
18.31
24
0.0711
0.0024
28
87.07
6
0.0015
3
0.0018
25
84.92
2
0.0011
11
0.0143
2
15.94
14
0.0088
9
0.0083
27
85.50
13
0.0048
18
0.0734
14
50.99
25
0.0734
18
0.0734
16
60.34
25
0.0734
21
0.1923
1
11.21
23
0.0368
21.36
20
0.0780
7
37.15
20
0.0276
68.30
15
0.0377
19
67.91
20
0.0276
2
15
59.28
14
0.0363
18
64.90
19
0.0275
1
14
56.55
5
0.0033
23
79.70
4
0.0013
2
9
40.52
7
0.0050
15
56.93
1
0.0010
1
7
35.33
10
0.0104
12
46.32
12
0.0047
2
19
69.82
7
0.0050
26
85.17
10
0.0027
1
13
54.67
12
0.0159
3
17.53
16
0.0102
2
11
47.51
13
0.0221
6
21.26
15
0.0095
1
—
—
—
—
8
42.60
28
0.0934
2
—
—
—
—
11
45.96
27
0.0826
1
—
—
—
—
21
73.31
9
0.0025
2
—
—
—
—
24
82.13
10
0.0027
1
8
38.03
1
0.0014
10
44.75
2
0.0011
2
4
20.02
2
0.0017
9
43.67
4
0.0013
1
—
—
—
—
29
87.76
22
0.0311
2
—
—
—
—
30
89.24
29
0.1680
1
—
—
—
—
22
75.14
18
0.0163
2
—
—
—
—
20
67.98
17
0.0155
1
16
59.38
6
0.0034
13
47.76
7
0.0024
2
—
—
—
—
17
63.19
7
0.0024
1
Table 57: Tabulation of the progression of identification timing and accuracy for Class C — Ten-Finger Plain-to-Plain. Letter refers to the participant’s
letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make.
The Time column shows the time used to perform a search over an enrollment set of 5 000 000 in seconds, but was originally recorded to microsecond
precision. The FNIR column shows FNIR for each submission at FPIR = 10−3 . NA indicates that the operations required to produce the value could
not be performed. — indicates that there was not a validated submission during that submission period. The number to the left of a value provides the
value’s column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
151
Second
Time
Sub. #
Third
FNIR
2
19.26
21
0.0826
2
7
28.17
20
1
11
46.42
6
2
16
70.86
1
1
12.49
2
8
29.59
1
13
55.63
2
15
68.61
1
3
20.17
2
5
1
19
2
1
Time
FNIR
4
25.25
16
0.0094
0.0822
5
25.91
15
0.0085
0.0028
17
65.97
4
0.0015
5
0.0023
30
86.39
7
0.0018
17
0.0285
1
9.52
18
0.0106
9
0.0121
14
57.02
12
0.0050
18
0.0536
13
55.63
25
0.0536
18
0.0536
18
68.61
25
0.0536
15
0.0239
2
11.67
24
0.0447
23.05
13
0.0195
6
30.61
21
0.0333
77.49
15
0.0239
28
84.14
20
0.0201
17
71.01
14
0.0237
29
84.26
19
0.0199
10
40.06
1
0.0015
24
79.04
1
0.0013
2
4
20.92
7
0.0032
9
40.53
2
0.0014
1
6
26.55
12
0.0168
8
36.09
13
0.0051
2
12
55.60
8
0.0045
20
72.62
9
0.0033
1
18
75.36
10
0.0123
7
31.42
17
0.0097
2
20
80.46
11
0.0160
3
19.52
14
0.0083
1
—
—
—
—
11
47.38
28
0.0783
2
—
—
—
—
12
48.71
27
0.0716
1
—
—
—
—
15
59.01
11
0.0034
2
—
—
—
—
19
72.55
9
0.0033
1
14
67.74
1
0.0015
26
83.25
5
0.0017
2
9
37.87
3
0.0019
27
83.35
2
0.0014
1
—
—
—
—
22
74.01
29
0.0860
2
—
—
—
—
21
72.95
30
0.2462
1
—
—
—
—
25
82.76
23
0.0358
2
—
—
—
—
23
74.39
22
0.0351
1
21
91.48
4
0.0022
10
40.94
5
0.0017
2
—
—
—
—
16
65.47
8
0.0019
1
Table 58: Tabulation of the progression of identification timing and accuracy for Class C — Ten-Finger Rolled-to-Rolled. Letter refers to the participant’s
letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make.
The Time column shows the time used to perform a search over an enrollment set of 5 000 000 in seconds, but was originally recorded to microsecond
precision. The FNIR column shows FNIR for each submission at FPIR = 10−3 . — indicates that there was not a validated submission during that
submission period. The number to the left of a value provides the value’s column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
152
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Second
Time
Sub. #
Third
FNIR
1
3
18.23
16
0.0979
2
4
19.43
17
1
8
39.02
6
2
14
58.60
1
5
20.29
2
10
55.20
1
13
58.39
2
16
61.82
1
1
11.46
2
2
1
20
2
1
Time
FNIR
7
36.55
17
0.0149
0.1124
5
33.93
18
0.0169
0.0048
12
56.38
6
0.0028
2
0.0034
25
76.00
3
0.0018
13
0.0383
1
10.81
16
0.0137
9
0.0164
16
59.20
12
0.0056
20
0.2514
15
58.39
27
0.2514
20
0.2514
17
61.82
27
0.2514
19
0.2393
3
25.05
24
0.0649
11.62
18
0.1455
6
34.47
23
0.0521
70.66
15
0.0429
18
65.03
20
0.0291
18
62.51
14
0.0411
19
65.69
19
0.0285
11
56.80
5
0.0039
26
76.98
2
0.0014
2
12
58.34
4
0.0037
21
66.90
1
0.0011
1
7
30.56
10
0.0174
8
42.13
13
0.0071
2
17
62.05
8
0.0053
27
79.37
7
0.0034
1
15
60.76
11
0.0251
4
27.43
15
0.0136
2
21
72.40
12
0.0358
2
18.62
14
0.0129
1
—
—
—
—
13
57.06
30
0.3067
2
—
—
—
—
14
57.90
27
0.2514
1
—
—
—
—
20
65.71
10
0.0041
2
—
—
—
—
24
75.98
8
0.0036
1
9
42.45
1
0.0028
10
49.07
4
0.0020
2
6
22.94
2
0.0034
11
49.72
5
0.0022
1
—
—
—
—
30
89.37
25
0.1017
2
—
—
—
—
29
89.34
26
0.2366
1
—
—
—
—
28
82.50
22
0.0378
2
—
—
—
—
23
73.46
21
0.0295
1
19
66.49
7
0.0051
9
48.91
11
0.0052
2
—
—
—
—
22
71.28
9
0.0039
Table 59: Tabulation of the progression of identification timing and accuracy for Class C — Ten-Finger Plain-to-Rolled. Letter refers to the participant’s
letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make.
The Time column shows the time used to perform a search over an enrollment set of 5 000 000 in seconds, but was originally recorded to microsecond
precision. The FNIR column shows FNIR for each submission at FPIR = 10−3 . — indicates that there was not a validated submission during that
submission period. The number to the left of a value provides the value’s column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
F
Enrollment Size
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
153
154
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
K
L
M
O
P
Q
S
T
U
V
RAM
Actual
Sub. #
On Disk
Reported
Finalized
Stored
FNIR @ FPIR = 10−3
1
7
0.18
10
0.16
8
0.17
8
0.16
30
0.1335
2
8
0.18
10
0.16
8
0.17
8
0.16
31
0.1337
1
21
0.61
18
0.33
29
1.25
18
0.34
1
0.0197
2
21
0.61
18
0.33
30
1.40
18
0.34
1
0.0197
1
30
1.12
27
0.50
23
0.50
27
0.50
16
0.0745
2
29
1.11
27
0.50
23
0.50
27
0.50
15
0.0723
1
6
0.18
7
0.11
10
0.17
10
0.17
25
0.1111
2
5
0.18
7
0.11
10
0.17
10
0.17
22
0.1082
1
20
0.57
29
0.50
5
0.15
5
0.15
24
0.1089
0.1086
2
19
0.57
29
0.50
5
0.15
5
0.15
23
1
11
0.33
14
0.29
12
0.29
14
0.29
32
0.1576
2
11
0.33
14
0.29
12
0.29
14
0.29
33
0.1607
1
23
0.70
35
0.68
31
1.54
35
0.68
7
0.0257
2
28
1.04
36
1.03
32
1.55
36
1.03
8
0.0278
1
9
0.32
12
0.27
14
0.31
12
0.28
18
0.0786
0.0712
2
9
0.32
12
0.27
14
0.31
12
0.28
14
1
35
3.87
25
0.47
35
4.60
25
0.48
21
0.0883
2
36
3.87
25
0.47
35
4.60
25
0.48
20
0.0875
1
31
1.14
9
0.15
7
0.16
7
0.16
11
0.0625
2
34
2.32
22
0.46
20
0.46
22
0.46
9
0.0351
1
3
0.14
5
0.11
3
0.13
3
0.13
35
0.2995
2
4
0.14
5
0.11
3
0.13
3
0.13
34
0.2921
1
15
0.35
16
0.30
16
0.32
16
0.31
19
0.0818
2
15
0.35
16
0.30
16
0.32
16
0.31
17
0.0766
1
27
0.78
3
0.03
1
0.03
1
0.03
29
0.1308
2
26
0.78
3
0.03
1
0.03
1
0.03
28
0.1272
1
13
0.34
1
0.03
27
1.07
33
0.68
3
0.0222
2
14
0.34
1
0.03
27
1.07
33
0.68
4
0.0226
1
24
0.72
23
0.47
21
0.47
23
0.47
10
0.0571
2
24
0.72
23
0.47
21
0.47
23
0.47
12
0.0650
1
2
0.01
33
0.60
25
0.91
31
0.60
36
NA
2
1
0.01
33
0.60
25
0.91
31
0.60
13
0.0685
1
33
1.70
31
0.53
33
1.96
29
0.54
27
0.1218
2
32
1.42
31
0.53
33
1.96
29
0.54
26
0.1178
1
18
0.39
20
0.38
18
0.39
20
0.38
6
0.0253
2
17
0.39
20
0.38
18
0.39
20
0.38
5
0.0252
Table 60: Tabulation of storage and RAM requirements for Class A — Left Index, with an enrollment set size of 100 000 subjects. Submissions were split into two groups. The
first group includes submissions that performed searches on average in less than 20 seconds, and the second includes those that took, on average, 20 seconds or longer. Letter
refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make
during the final submission period. All values are reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. Actual RAM refers to the sum of the resident set sizes of
the stage one identification processes over all compute nodes after returning from the identification stage one initialization method. Reported RAM is the sum of the predicted
RAM consumption returned from the enrollment method when enrolling images for the enrollment set. Finalized Size is the amount of disk storage space used in the finalized
enrollment set directory after the finalization method returned. Stored Size is the actual amount of storage space used to hold the individual enrollment templates on disk,
determined by summing the number of bytes written by the FpVTE test driver after each enrollment. The number to the left of a value provides the value’s column-wise
ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 7 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
K
L
M
O
P
Q
S
T
U
V
RAM
Actual
Sub. #
155
On Disk
Reported
Finalized
Stored
FNIR @ FPIR = 10−3
1
7
0.17
10
0.15
8
0.16
8
0.16
30
0.1132
2
8
0.18
10
0.15
8
0.16
8
0.16
29
0.1124
1
21
0.61
18
0.33
29
1.25
18
0.33
1
0.0190
2
21
0.61
18
0.33
30
1.40
18
0.33
1
0.0190
1
29
1.07
27
0.48
23
0.48
27
0.48
15
0.0630
2
30
1.07
27
0.48
23
0.48
27
0.48
14
0.0624
1
6
0.17
7
0.11
10
0.17
10
0.16
25
0.0933
2
5
0.17
7
0.11
10
0.17
10
0.16
22
0.0903
1
19
0.57
29
0.48
5
0.15
5
0.15
24
0.0910
0.0909
2
20
0.57
29
0.48
5
0.15
5
0.15
23
1
14
0.33
14
0.29
14
0.29
14
0.29
32
0.1230
2
13
0.33
14
0.29
14
0.29
14
0.29
33
0.1249
1
23
0.69
35
0.67
31
1.54
35
0.67
5
0.0215
2
28
1.01
36
0.99
32
1.55
36
0.99
3
0.0214
1
10
0.31
12
0.26
12
0.29
12
0.27
20
0.0708
0.0643
2
9
0.31
12
0.26
12
0.29
12
0.27
16
1
36
3.87
25
0.46
35
4.60
25
0.47
18
0.0682
2
35
3.87
25
0.46
35
4.60
25
0.47
19
0.0685
1
31
1.14
9
0.15
7
0.16
7
0.15
12
0.0505
2
34
2.32
22
0.44
20
0.45
22
0.45
9
0.0295
1
3
0.14
5
0.11
3
0.13
3
0.12
36
0.2615
2
3
0.14
5
0.11
3
0.13
3
0.12
35
0.2526
1
15
0.34
16
0.29
16
0.30
16
0.30
21
0.0776
2
16
0.34
16
0.29
16
0.30
16
0.30
17
0.0675
1
26
0.76
3
0.03
1
0.03
1
0.03
31
0.1133
2
27
0.76
3
0.03
1
0.03
1
0.03
28
0.1100
1
11
0.33
1
0.03
27
1.03
33
0.66
6
0.0218
2
11
0.33
1
0.03
27
1.03
33
0.66
3
0.0214
1
24
0.71
23
0.46
21
0.46
23
0.46
10
0.0442
2
25
0.71
23
0.46
21
0.46
23
0.46
11
0.0503
1
1
0.01
33
0.57
25
0.87
31
0.57
34
0.1929
2
1
0.01
33
0.57
25
0.87
31
0.57
13
0.0562
1
33
1.56
31
0.52
33
1.79
29
0.52
26
0.0996
2
32
1.30
31
0.52
33
1.79
29
0.52
27
0.1007
1
18
0.39
20
0.38
18
0.38
20
0.38
8
0.0223
2
17
0.39
20
0.38
18
0.38
20
0.38
7
0.0222
Table 61: Tabulation of storage and RAM requirements for Class A — Right Index, with an enrollment set size of 100 000 subjects. Submissions were split into two groups.
The first group includes submissions that performed searches on average in less than 20 seconds, and the second includes those that took, on average, 20 seconds or longer.
Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could
make during the final submission period. All values are reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. Actual RAM refers to the sum of the resident set
sizes of the stage one identification processes over all compute nodes after returning from the identification stage one initialization method. Reported RAM is the sum of the
predicted RAM consumption returned from the enrollment method when enrolling images for the enrollment set. Finalized Size is the amount of disk storage space used in the
finalized enrollment set directory after the finalization method returned. Stored Size is the actual amount of storage space used to hold the individual enrollment templates on
disk, determined by summing the number of bytes written by the FpVTE test driver after each enrollment. The number to the left of a value provides the value’s column-wise
ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 8 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
156
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
K
L
M
O
P
Q
S
T
U
V
RAM
Actual
Sub. #
On Disk
Reported
Finalized
Stored
FNIR @ FPIR = 10−3
1
8
4.87
10
4.66
8
4.78
8
4.69
26
0.0368
2
7
4.87
10
4.66
8
4.78
8
4.69
28
0.0374
1
24
18.58
18
10.30
30
40.04
18
10.31
4
0.0030
2
24
18.58
18
10.30
31
44.68
18
10.31
4
0.0030
1
30
33.40
27
14.95
23
15.02
27
14.96
15
0.0207
2
31
33.41
27
14.95
23
15.02
27
14.96
14
0.0202
1
5
4.86
5
3.21
10
4.86
10
4.78
29
0.0386
2
5
4.86
5
3.21
10
4.86
10
4.78
30
0.0412
1
23
16.38
31
15.63
6
4.75
6
4.67
31
0.0515
0.0311
2
22
16.38
31
15.63
6
4.75
6
4.67
20
1
16
9.36
16
9.19
16
9.32
16
9.19
33
0.0686
2
15
9.36
16
9.19
16
9.32
16
9.19
32
0.0684
1
21
15.83
33
15.68
25
24.61
31
15.68
8
0.0058
2
29
30.83
36
30.68
32
46.83
36
30.68
4
0.0030
1
9
8.30
12
8.07
12
8.68
12
8.15
10
0.0143
0.0143
2
10
8.30
12
8.07
12
8.68
12
8.15
10
1
36
61.81
25
14.74
35
73.67
25
14.80
24
0.0360
2
35
61.81
25
14.74
35
73.67
25
14.80
19
0.0286
1
26
18.76
9
4.55
5
4.68
5
4.61
12
0.0146
2
34
53.98
22
13.64
20
13.73
22
13.66
9
0.0072
1
3
3.75
5
3.21
3
3.75
3
3.68
35
NA
2
4
3.76
5
3.21
3
3.75
3
3.68
34
NA
1
12
9.05
14
8.82
14
8.84
14
8.90
17
0.0229
2
11
9.05
14
8.82
14
8.84
14
8.90
16
0.0214
1
28
21.42
3
0.86
1
0.89
1
0.83
27
0.0370
2
27
21.41
3
0.86
1
0.89
1
0.83
21
0.0333
1
14
9.14
1
0.82
28
30.63
34
20.48
1
0.0027
2
13
9.14
1
0.82
28
30.63
34
20.48
1
0.0027
1
19
14.82
23
14.57
21
14.63
23
14.57
18
0.0281
2
19
14.82
23
14.57
21
14.63
23
14.57
13
0.0195
1
1
0.01
34
17.99
26
27.40
32
18.02
36
NA
2
1
0.01
34
17.99
26
27.40
32
18.02
25
0.0366
1
33
44.63
29
15.30
33
51.84
29
15.46
22
0.0336
2
32
37.35
29
15.30
33
51.84
29
15.46
23
0.0358
1
18
11.77
20
11.69
18
11.77
20
11.71
7
0.0034
2
17
11.77
20
11.69
18
11.77
20
11.71
3
0.0028
Table 62: Tabulation of storage and RAM requirements for Class A — Left and Right Index, with an enrollment set size of 1 600 000 subjects. Submissions were split into two
groups. The first group includes submissions that performed searches on average in less than 20 seconds, and the second includes those that took, on average, 20 seconds or
longer. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant
could make during the final submission period. All values are reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. Actual RAM refers to the sum of the resident
set sizes of the stage one identification processes over all compute nodes after returning from the identification stage one initialization method. Reported RAM is the sum of the
predicted RAM consumption returned from the enrollment method when enrolling images for the enrollment set. Finalized Size is the amount of disk storage space used in the
finalized enrollment set directory after the finalization method returned. Stored Size is the actual amount of storage space used to hold the individual enrollment templates on
disk, determined by summing the number of bytes written by the FpVTE test driver after each enrollment. The number to the left of a value provides the value’s column-wise
ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 9 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
RAM
Actual
Sub. #
3
46.49
2
4
1
12
2
13
1
On Disk
Reported
Finalized
Stored
FNIR @ FPIR = 10−3
2
45.98
30
NA
46.34
2
45.98
29
NA
86.78
18
86.59
6
0.0020
85.95
17
85.76
2
0.0012
1
30.00
1
29.89
16
0.0043
176.65
26
176.79
28
176.67
7
0.0024
11
53.02
12
77.01
13
76.88
27
0.0591
11
53.02
12
77.01
13
76.88
27
0.0591
0.0062
4
45.92
46.49
4
79.43
16
79.43
15
14
79.73
2
28
1
2
1
157
2
46.34
45.92
2
84.51
18
83.67
15
3
29.86
317.95
30
10
77.02
10
77.02
1
25
156.12
26
152.16
4
47.21
4
47.07
18
2
26
156.12
26
152.16
4
47.21
4
47.07
14
0.0040
1
15
86.46
17
84.85
16
86.42
15
84.85
23
0.0203
2
16
86.46
17
84.85
16
86.42
15
84.85
24
0.0204
1
5
49.68
8
49.40
14
82.58
8
49.40
2
0.0012
2
21
108.71
23
108.43
23
133.80
23
108.43
1
0.0009
0.0049
1
18
101.00
21
101.67
21
100.73
21
100.75
17
2
17
101.00
21
101.67
21
100.73
21
100.75
11
0.0033
1
22
119.79
7
47.80
7
47.92
7
47.77
10
0.0031
2
27
177.48
6
46.73
6
47.56
6
47.42
11
0.0033
1
6
57.53
9
53.02
8
57.52
9
57.39
26
0.0543
2
6
57.53
9
53.02
8
57.52
9
57.39
25
0.0515
1
19
101.00
19
101.67
19
100.73
19
100.75
15
0.0041
2
20
101.00
19
101.67
19
100.73
19
100.75
13
0.0035
1
1
7.54
1
8.87
27
232.47
29
222.31
2
0.0012
2
2
7.54
1
8.87
27
232.47
29
222.31
2
0.0012
1
24
150.35
24
147.31
24
147.42
24
147.31
20
0.0108
2
23
150.35
24
147.31
24
147.42
24
147.31
21
0.0136
1
29
440.68
28
155.98
29
512.65
26
157.35
19
0.0099
2
30
540.60
29
170.16
30
625.07
27
171.53
22
0.0141
1
8
63.53
13
63.39
10
63.52
11
63.41
9
0.0027
2
9
63.53
13
63.39
10
63.52
11
63.41
7
0.0024
Table 63: Tabulation of storage and RAM requirements for Class B — Identification Flats, with an enrollment set size of 3 000 000 subjects. Letter refers
to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant
could make during the final submission period. All values are reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. Actual RAM refers to the
sum of the resident set sizes of the stage one identification processes over all compute nodes after returning from the identification stage one initialization
method. Reported RAM is the sum of the predicted RAM consumption returned from the enrollment method when enrolling images for the enrollment
set. Finalized Size is the amount of disk storage space used in the finalized enrollment set directory after the finalization method returned. Stored Size
is the actual amount of storage space used to hold the individual enrollment templates on disk, determined by summing the number of bytes written
by the FpVTE test driver after each enrollment. The number to the left of a value provides the value’s column-wise ranking, with the best performance
shaded in green and the worst in pink. For reference, the FNIR values from Table 13 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
158
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
RAM
Actual
Sub. #
On Disk
Reported
Finalized
Stored
FNIR @ FPIR = 10−3
1
9
92.86
11
96.24
10
92.63
11
96.33
30
NA
2
10
92.86
11
96.24
10
92.63
11
96.33
24
0.0711
1
16
132.37
19
155.88
19
159.68
19
159.37
6
0.0015
2
15
132.37
18
153.13
18
156.93
18
156.62
2
0.0011
1
14
127.39
3
57.47
1
57.71
1
57.52
14
0.0088
2
28
573.25
30
318.66
26
318.89
28
318.70
13
0.0048
1
7
90.93
6
82.27
6
90.92
7
90.70
25
0.0734
2
6
90.93
6
82.27
6
90.92
7
90.70
25
0.0734
0.0368
1
25
274.35
26
267.49
4
82.40
4
82.17
23
2
26
274.35
26
267.49
4
82.40
4
82.17
20
0.0276
1
17
144.02
16
142.06
14
143.97
16
142.06
20
0.0276
2
18
144.02
16
142.06
14
143.97
16
142.06
19
0.0275
1
4
84.42
10
83.96
16
144.38
6
83.96
4
0.0013
2
11
110.22
13
109.76
17
153.35
13
109.76
1
0.0010
0.0047
1
20
181.00
22
182.11
20
182.47
22
180.58
12
2
21
181.00
23
182.11
23
182.47
23
180.58
10
0.0027
1
23
192.19
4
72.47
2
72.67
2
72.43
16
0.0102
2
27
274.95
4
72.47
3
72.67
2
72.43
15
0.0095
1
5
90.93
8
82.27
8
90.92
9
90.70
28
0.0934
2
8
90.93
8
82.27
8
90.92
9
90.70
27
0.0826
1
19
181.00
20
182.11
21
182.47
20
180.58
9
0.0025
2
22
181.00
20
182.11
21
182.47
20
180.58
10
0.0027
1
2
13.33
1
15.77
27
412.86
29
395.06
2
0.0011
2
1
13.33
1
15.77
27
412.86
29
395.06
4
0.0013
1
24
260.37
24
257.08
25
257.27
24
257.09
22
0.0311
2
3
61.04
24
257.08
24
257.27
24
257.09
29
0.1680
1
29
811.45
28
293.25
29
941.34
26
295.54
18
0.0163
2
29
811.45
28
293.25
29
941.34
26
295.54
17
0.0155
1
13
121.80
14
121.56
12
121.79
14
121.60
7
0.0024
2
12
121.79
14
121.56
12
121.79
14
121.60
7
0.0024
Table 64: Tabulation of storage and RAM requirements for Class C — Ten-Finger Plain-to-Plain, with an enrollment set size of 5 000 000 subjects. Letter
refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each
participant could make during the final submission period. All values are reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. Actual RAM
refers to the sum of the resident set sizes of the stage one identification processes over all compute nodes after returning from the identification stage one
initialization method. Reported RAM is the sum of the predicted RAM consumption returned from the enrollment method when enrolling images for the
enrollment set. Finalized Size is the amount of disk storage space used in the finalized enrollment set directory after the finalization method returned.
Stored Size is the actual amount of storage space used to hold the individual enrollment templates on disk, determined by summing the number of
bytes written by the FpVTE test driver after each enrollment. The number to the left of a value provides the value’s column-wise ranking, with the best
performance shaded in green and the worst in pink. For reference, the FNIR values from Table 14 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
RAM
Actual
Sub. #
159
On Disk
Reported
Finalized
Stored
FNIR @ FPIR = 10−3
1
14
183.36
14
190.50
14
183.11
14
190.59
16
0.0094
2
15
183.36
14
190.50
14
183.11
14
190.59
15
0.0085
1
10
132.40
22
302.53
18
303.52
24
303.21
4
0.0015
2
9
132.37
27
309.65
19
310.65
25
310.33
7
0.0018
1
16
191.66
3
95.45
1
95.69
1
95.50
18
0.0106
2
30
930.48
30
528.40
26
528.63
28
528.45
12
0.0050
1
7
130.29
5
118.45
4
130.29
5
130.07
25
0.0536
2
6
130.29
5
118.45
4
130.29
5
130.07
25
0.0536
0.0447
1
20
261.29
18
254.66
2
97.99
2
97.76
24
2
19
261.29
18
254.66
2
97.99
2
97.76
21
0.0333
1
13
144.02
10
143.46
8
143.97
10
143.46
20
0.0201
2
12
144.02
10
143.46
8
143.97
10
143.46
19
0.0199
1
4
113.80
4
113.34
12
153.41
4
113.34
1
0.0013
2
11
137.03
9
136.57
13
153.69
9
136.57
2
0.0014
0.0051
1
22
303.13
24
304.24
21
355.15
21
302.68
13
2
24
303.13
23
304.24
20
355.15
20
302.68
9
0.0033
1
21
280.49
12
151.41
10
151.61
12
151.37
17
0.0097
2
26
367.82
12
151.41
11
151.61
12
151.37
14
0.0083
1
8
130.29
7
118.45
6
130.29
7
130.07
28
0.0783
2
5
130.29
7
118.45
6
130.29
7
130.07
27
0.0716
1
25
303.13
25
304.24
22
355.15
22
302.68
11
0.0034
2
23
303.13
25
304.24
22
355.15
22
302.68
9
0.0033
1
1
20.22
1
25.61
27
666.73
29
641.13
5
0.0017
2
2
20.22
1
25.61
27
666.73
29
641.13
2
0.0014
1
27
382.88
28
381.17
25
381.36
26
381.17
29
0.0860
2
3
78.28
28
381.17
24
381.36
26
381.17
30
0.2462
1
28
806.17
20
257.87
29
937.76
18
260.17
23
0.0358
2
28
806.17
20
257.87
29
937.76
18
260.17
22
0.0351
1
17
234.03
16
233.80
16
234.03
16
233.84
5
0.0017
2
18
234.03
16
233.80
16
234.03
16
233.84
8
0.0019
Table 65: Tabulation of storage and RAM requirements for Class C — Ten-Finger Rolled-to-Rolled, with an enrollment set size of 5 000 000 subjects. Letter
refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each
participant could make during the final submission period. All values are reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. Actual RAM
refers to the sum of the resident set sizes of the stage one identification processes over all compute nodes after returning from the identification stage one
initialization method. Reported RAM is the sum of the predicted RAM consumption returned from the enrollment method when enrolling images for the
enrollment set. Finalized Size is the amount of disk storage space used in the finalized enrollment set directory after the finalization method returned.
Stored Size is the actual amount of storage space used to hold the individual enrollment templates on disk, determined by summing the number of
bytes written by the FpVTE test driver after each enrollment. The number to the left of a value provides the value’s column-wise ranking, with the best
performance shaded in green and the worst in pink. For reference, the FNIR values from Table 15 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
160
G
G.1
FPVTE – F INGERPRINT M ATCHING
Search Template Sizes
Mean Values
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
K
L
M
O
P
Q
S
T
U
V
Left Index
Size
Sub. #
161
Right Index
FNIR
Size
Left and Right Index
Size
FNIR
FNIR
1
5
1.49
30
0.1335
5
1.51
30
0.1132
5
2.98
26
0.0368
2
5
1.49
31
0.1337
5
1.51
29
0.1124
5
2.98
28
0.0374
1
18
3.31
1
0.0197
18
3.34
1
0.0190
18
6.65
4
0.0030
2
18
3.31
1
0.0197
18
3.34
1
0.0190
18
6.65
4
0.0030
1
27
4.83
16
0.0745
27
4.89
15
0.0630
27
9.71
15
0.0207
2
27
4.83
15
0.0723
27
4.89
14
0.0624
27
9.71
14
0.0202
1
8
1.54
25
0.1111
8
1.55
25
0.0933
7
3.00
29
0.0386
2
8
1.54
22
0.1082
8
1.55
22
0.0903
7
3.00
30
0.0412
1
10
2.51
24
0.1089
10
2.52
24
0.0910
10
5.02
31
0.0515
2
10
2.51
23
0.1086
10
2.52
23
0.0909
10
5.02
20
0.0311
1
16
3.00
32
0.1576
16
3.00
32
0.1230
16
5.99
33
0.0686
2
16
3.00
33
0.1607
16
3.00
33
0.1249
16
5.99
32
0.0684
1
35
6.83
7
0.0257
35
6.87
5
0.0215
29
10.19
8
0.0058
2
36
10.07
8
0.0278
36
10.18
3
0.0214
36
20.17
4
0.0030
1
12
2.68
18
0.0786
12
2.72
20
0.0708
12
5.26
10
0.0143
2
12
2.68
14
0.0712
12
2.72
16
0.0643
12
5.26
10
0.0143
1
25
4.82
21
0.0883
25
4.85
18
0.0682
25
9.67
24
0.0360
2
25
4.82
20
0.0875
25
4.85
19
0.0685
25
9.67
19
0.0286
1
7
1.52
11
0.0625
7
1.53
12
0.0505
9
3.01
12
0.0146
2
22
4.47
9
0.0351
22
4.51
9
0.0295
22
8.93
9
0.0072
1
3
1.21
35
0.2995
3
1.18
36
0.2615
3
2.31
35
NA
2
3
1.21
34
0.2921
3
1.18
35
0.2526
3
2.31
34
NA
1
14
2.93
19
0.0818
14
2.98
21
0.0776
14
5.77
17
0.0229
2
14
2.93
17
0.0766
14
2.98
17
0.0675
14
5.77
16
0.0214
1
1
0.28
29
0.1308
1
0.28
31
0.1133
1
0.53
27
0.0370
2
1
0.28
28
0.1272
1
0.28
28
0.1100
1
0.53
21
0.0333
1
33
6.52
3
0.0222
33
6.57
6
0.0218
34
13.08
1
0.0027
2
33
6.52
4
0.0226
33
6.57
3
0.0214
34
13.08
1
0.0027
1
23
4.74
10
0.0571
23
4.77
10
0.0442
23
9.52
18
0.0281
2
23
4.74
12
0.0650
23
4.77
11
0.0503
23
9.52
13
0.0195
1
31
6.00
36
NA
31
6.09
34
0.1929
32
12.09
36
NA
2
31
6.00
13
0.0685
31
6.09
13
0.0562
32
12.09
25
0.0366
1
29
5.17
27
0.1218
29
5.20
26
0.0996
30
10.36
22
0.0336
2
29
5.17
26
0.1178
29
5.20
27
0.1007
30
10.36
23
0.0358
1
20
3.70
6
0.0253
20
3.72
8
0.0223
20
7.41
7
0.0034
2
20
3.70
5
0.0252
20
3.72
7
0.0222
20
7.41
3
0.0028
Table 66: Tabulation of mean search template sizes for Class A. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is
an identifier used to differentiate between the two submissions each participant could make. Size values indicate the mean kilobytes used to store a
search template on disk for a single subject, where 1 kB is equal to 1 024 bytes. The FNIR column shows FNIR for each submission at FPIR = 10−3 . NA
indicates that the operations required to produce the value could not be performed. The number to the left of a value provides the value’s column-wise
ranking, with the best performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
162
24
0.0072
0.0392
0.0403
16
4
4
22.39
13.24
13.24
6
29
30
0.0031
NA
NA
18
2
2
30.53
16.13
16.13
6
29
30
0.0020
NA
NA
Identification Flats
Size
FNIR
6.70
6
23
Left and Right Slap
Size
FNIR
6.70
FNIR
4
11.31
Right Slap
0.0654
4
Size
22
16
FNIR
6.59
0.0647
Left Slap
4
0.0163
Size
1
6
21
0.0024
6.59
0.0591
11.27
7
0.0591
4
27
0.0062
16
62.06
27
0.0040
2
26.80
18
0.0203
1
28
26.80
14
0.0012
13
26.45
23
0.0043
0.0049
13
26.45
2
0.0910
11
29.66
16
10
0.0901
11
30.22
28
0.0106
15
10.48
46.65
26
0.0084
1
19.21
18
0.0349
17
28
19.21
17
0.0204
0.0024
11
20.53
23
24
0.0012
0.0063
0.0083
11
20.53
2
5
0.1222
13
23.77
29.66
1
15
7
0.1220
13
15
17.69
7.78
29
0.0212
17
6
22.24
28
0.0198
0.0361
23
0.0049
0.0009
1
9.73
23.53
18
0.0641
24
0.0022
17
15
28
9.73
16
3
38.41
0.0052
11
10.32
25
23.77
1
35.39
0.0151
0.0187
11
10.32
17
14.37
21
2
0.1684
13
11.90
8
0.0068
0.0015
13
7
0.1681
13
0.0647
23
16
3.93
29
0.0371
17
26
0.0058
28.60
11.24
28
0.0325
5
25.51
1
9.66
23.22
18
0.0998
11.90
1
21
15
28
9.66
17
17
8.97
0.0156
0.0045
0.0142
11
10.25
23
10
14
0.0259
2
11
10.25
0.1008
23
14.42
5
1
13
11.92
24
0.0116
12.94
13
2
13
4
19
3.88
1
17
11.92
1
0.0287
0.0094
11.18
2
17
8.92
15
1
1
10
14.25
15
2
23
12.76
2
1
21
27
14
0.0057
0.0882
0.0904
0.0062
29
19
19
7
7
4
51.62
78.12
78.12
35.39
35.39
19.93
19.93
16.79
19
21
20
2
2
13
15
25
26
11
0.0027
0.0141
0.0099
0.0136
0.0108
0.0012
0.0012
0.0035
0.0041
0.0515
0.0543
0.0033
1
12.03
25
0.0051
29
51.62
22
0.0024
0.0033
14.11
13
0.0021
24
55.81
9
11
2
14.11
11
0.0022
24
60.97
7
21
6
25.51
2
0.0160
26
22.27
9
0.0202
6
25.51
3
0.0190
27
22.27
21
0.1259
19
58.19
21
0.0139
9
10
17
0.1155
19
58.19
22
0.0124
9
19
30
0.0142
29
40.83
20
0.0036
10
6.08
27
0.0132
29
40.83
19
0.0036
21
7.13
12
0.0057
24
41.34
7
0.0031
2
7.13
11
0.0057
24
42.94
7
10
6
12.94
3
0.0369
26
17.22
35.39
0.0276
6
12.94
3
0.0381
27
17.22
16.92
0.1736
21
29.30
21
0.0266
9
5
14
0.1634
21
29.30
22
0.0273
9
0.0047
30
0.0257
29
20.56
19
0.0106
0.0054
6.02
27
0.0254
29
20.56
20
0.0110
12
7.08
12
0.0098
24
20.73
8
25.51
2
7.08
11
0.0099
24
21.60
9
12.15
1
6
12.76
2
0.1089
26
8.65
3
2
6
12.76
3
0.1133
27
8.65
0.0126
1
19
29.00
25
0.0500
8
0.0167
2
19
29.00
26
0.0461
8
15
1
29
20.36
20
0.0192
6.14
2
29
20.36
19
0.0190
12.94
1
24
20.70
9
3
2
24
21.43
8
0.0236
1
26
8.61
0.0288
2
27
8.61
16
1
8
6.08
2
8
12.76
1
3
2
2
2
1
Sub. #
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Table 67: Tabulation of mean search template sizes for Class B. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between
the two submissions each participant could make. Size values indicate the mean kilobytes used to store a search template on disk for a single subject, where 1 kB is equal to 1 024 bytes. The
FNIR column shows FNIR for each submission at FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. The number to the left of a value
provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Ten-Finger Plain-to-Plain
Size
FNIR
Sub. #
163
Ten-Finger Rolled-to-Rolled
Size
FNIR
9
20.91
30
NA
14
40.15
16
0.0094
2
9
20.91
24
0.0711
14
40.15
15
0.0085
1
19
33.66
6
0.0015
24
63.63
4
0.0015
2
18
33.16
2
0.0011
25
65.36
7
0.0018
1
1
12.10
14
0.0088
1
19.95
18
0.0106
2
28
67.07
13
0.0048
28
110.22
12
0.0050
1
5
19.45
25
0.0734
3
27.41
25
0.0536
2
5
19.45
25
0.0734
3
27.41
25
0.0536
1
14
27.52
23
0.0368
10
30.76
24
0.0447
2
14
27.52
20
0.0276
10
30.76
21
0.0333
1
16
29.72
20
0.0276
8
29.96
20
0.0201
2
16
29.72
19
0.0275
8
29.96
19
0.0199
1
4
17.76
4
0.0013
2
23.76
1
0.0013
2
11
23.15
1
0.0010
7
28.49
2
0.0014
1
23
38.56
12
0.0047
23
63.60
13
0.0051
2
22
38.56
10
0.0027
20
63.60
9
0.0033
1
2
15.74
16
0.0102
12
32.38
17
0.0097
2
2
15.74
15
0.0095
12
32.38
14
0.0083
1
5
19.45
28
0.0934
3
27.41
28
0.0783
2
5
19.45
27
0.0826
3
27.41
27
0.0716
1
20
38.56
9
0.0025
21
63.60
11
0.0034
2
20
38.56
10
0.0027
21
63.60
9
0.0033
1
29
83.85
2
0.0011
29
134.42
5
0.0017
2
29
83.85
4
0.0013
29
134.42
2
0.0014
1
24
54.12
22
0.0311
26
80.04
29
0.0860
2
24
54.12
29
0.1680
26
80.04
30
0.2462
1
26
62.80
18
0.0163
18
54.62
23
0.0358
2
26
62.80
17
0.0155
18
54.62
22
0.0351
1
12
25.78
7
0.0024
16
48.74
5
0.0017
2
12
25.78
7
0.0024
16
48.74
8
0.0019
1
Table 68: Tabulation of mean search template sizes for Class C. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is
an identifier used to differentiate between the two submissions each participant could make. Size values indicate the mean kilobytes used to store a
search template on disk for a single subject, where 1 kB is equal to 1 024 bytes. The FNIR column shows FNIR for each submission at FPIR = 10−3 . NA
indicates that the operations required to produce the value could not be performed. The number to the left of a value provides the value’s column-wise
ranking, with the best performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
164
G.2
FPVTE – F INGERPRINT M ATCHING
Median Values
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
K
L
M
O
P
Q
S
T
U
V
Left Index
Size
Sub. #
165
Right Index
FNIR
Size
Left and Right Index
Size
FNIR
FNIR
1
5
1.49
30
0.1335
5
1.51
30
0.1132
5
2.94
26
0.0368
2
5
1.49
31
0.1337
5
1.51
29
0.1124
5
2.94
28
0.0374
1
18
3.27
1
0.0197
18
3.30
1
0.0190
18
6.57
4
0.0030
2
18
3.27
1
0.0197
18
3.30
1
0.0190
18
6.57
4
0.0030
1
27
4.84
16
0.0745
27
4.91
15
0.0630
27
9.71
15
0.0207
2
27
4.84
15
0.0723
27
4.91
14
0.0624
27
9.71
14
0.0202
1
8
1.52
25
0.1111
8
1.53
25
0.0933
8
2.97
29
0.0386
2
8
1.52
22
0.1082
8
1.53
22
0.0903
8
2.97
30
0.0412
1
10
2.35
24
0.1089
10
2.36
24
0.0910
10
4.81
31
0.0515
2
10
2.35
23
0.1086
10
2.36
23
0.0909
10
4.81
20
0.0311
1
16
3.00
32
0.1576
16
3.00
32
0.1230
16
6.00
33
0.0686
2
16
3.00
33
0.1607
16
3.00
33
0.1249
16
6.00
32
0.0684
1
35
6.84
7
0.0257
35
6.88
5
0.0215
31
10.19
8
0.0058
2
36
10.02
8
0.0278
36
10.12
3
0.0214
36
20.05
4
0.0030
1
12
2.65
18
0.0786
12
2.68
20
0.0708
12
5.19
10
0.0143
2
12
2.65
14
0.0712
12
2.68
16
0.0643
12
5.19
10
0.0143
1
25
4.76
21
0.0883
25
4.79
18
0.0682
25
9.55
24
0.0360
2
25
4.76
20
0.0875
25
4.79
19
0.0685
25
9.55
19
0.0286
1
7
1.50
11
0.0625
7
1.51
12
0.0505
7
2.97
12
0.0146
2
22
4.41
9
0.0351
22
4.45
9
0.0295
22
8.81
9
0.0072
1
3
1.20
35
0.2995
3
1.17
36
0.2615
3
2.28
35
NA
2
3
1.20
34
0.2921
3
1.17
35
0.2526
3
2.28
34
NA
1
14
2.90
19
0.0818
14
2.94
21
0.0776
14
5.69
17
0.0229
2
14
2.90
17
0.0766
14
2.94
17
0.0675
14
5.69
16
0.0214
1
1
0.28
29
0.1308
1
0.28
31
0.1133
1
0.53
27
0.0370
2
1
0.28
28
0.1272
1
0.28
28
0.1100
1
0.53
21
0.0333
1
33
6.46
3
0.0222
33
6.51
6
0.0218
34
12.94
1
0.0027
2
33
6.46
4
0.0226
33
6.51
3
0.0214
34
12.94
1
0.0027
1
23
4.69
10
0.0571
23
4.71
10
0.0442
23
9.40
18
0.0281
2
23
4.69
12
0.0650
23
4.71
11
0.0503
23
9.40
13
0.0195
1
31
5.81
36
NA
31
5.89
34
0.1929
32
11.73
36
NA
2
31
5.81
13
0.0685
31
5.89
13
0.0562
32
11.73
25
0.0366
1
29
5.01
27
0.1218
29
5.04
26
0.0996
29
10.05
22
0.0336
2
29
5.01
26
0.1178
29
5.04
27
0.1007
29
10.05
23
0.0358
1
20
3.63
6
0.0253
20
3.65
8
0.0223
20
7.27
7
0.0034
2
20
3.63
5
0.0252
20
3.65
7
0.0222
20
7.27
3
0.0028
Table 69: Tabulation of median search template sizes for Class A. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is
an identifier used to differentiate between the two submissions each participant could make. Size values indicate the median kilobytes used to store a
search template on disk for a single subject, where 1 kB is equal to 1 024 bytes. The FNIR column shows FNIR for each submission at FPIR = 10−3 . NA
indicates that the operations required to produce the value could not be performed. The number to the left of a value provides the value’s column-wise
ranking, with the best performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
166
24
0.0072
0.0392
0.0403
16
4
4
22.48
13.18
13.18
6
29
30
0.0031
NA
NA
18
2
2
30.57
16.03
16.03
6
29
30
0.0020
NA
NA
Identification Flats
Size
FNIR
6.67
6
23
Left and Right Slap
Size
FNIR
6.67
FNIR
4
11.35
Right Slap
0.0654
4
Size
22
16
FNIR
6.56
0.0647
Left Slap
4
0.0163
Size
1
6
21
0.0024
6.56
0.0591
11.30
7
0.0591
4
27
0.0062
16
62.33
27
0.0040
2
26.48
18
0.0203
1
28
26.48
14
0.0012
13
25.93
23
0.0043
0.0049
13
25.93
2
0.0910
11
30.00
16
10
0.0901
11
30.17
28
0.0106
15
10.43
46.93
26
0.0084
1
18.78
18
0.0349
17
28
18.78
17
0.0204
0.0024
11
20.12
23
24
0.0012
0.0063
0.0083
11
20.12
2
5
0.1222
13
24.00
30.00
1
15
7
0.1220
13
15
17.87
7.78
29
0.0212
17
6
22.24
28
0.0198
0.0361
23
0.0049
0.0009
1
9.49
23.68
18
0.0641
24
0.0022
17
15
28
9.49
16
3
38.44
0.0052
11
10.07
25
24.00
1
35.37
0.0151
0.0187
11
10.07
17
14.52
19
2
0.1684
13
12.00
8
0.0068
0.0015
13
7
0.1681
13
0.0647
23
16
3.91
29
0.0371
17
26
0.0058
28.63
11.24
28
0.0325
5
25.46
1
9.41
23.35
18
0.0998
12.00
1
19
15
28
9.41
17
17
9.05
0.0156
0.0045
0.0142
11
9.99
23
10
14
0.0259
2
11
9.99
0.1008
23
14.42
5
1
13
12.00
24
0.0116
12.93
13
2
13
4
19
3.89
1
17
12.00
1
0.0287
0.0094
11.18
2
17
9.00
15
1
1
10
14.26
15
2
23
12.73
2
1
19
27
14
0.0057
0.0882
0.0904
0.0062
29
19
19
7
7
4
51.27
78.05
78.05
35.37
35.37
19.73
19.73
16.74
19
21
20
2
2
13
15
25
26
11
0.0027
0.0141
0.0099
0.0136
0.0108
0.0012
0.0012
0.0035
0.0041
0.0515
0.0543
0.0033
1
12.00
25
0.0051
29
51.27
22
0.0024
0.0033
13.82
13
0.0021
24
56.14
9
11
2
13.82
11
0.0022
24
60.55
7
19
6
25.46
2
0.0160
26
22.46
9
0.0202
6
25.46
3
0.0190
27
22.46
19
0.1259
19
58.20
21
0.0139
9
10
17
0.1155
19
58.20
22
0.0124
9
19
30
0.0142
29
40.31
20
0.0036
10
6.06
27
0.0132
29
40.31
19
0.0036
19
6.97
12
0.0057
24
41.58
7
0.0031
2
6.97
11
0.0057
24
42.60
7
10
6
12.93
3
0.0369
26
17.26
35.37
0.0276
6
12.93
3
0.0381
27
17.26
16.86
0.1736
19
29.30
21
0.0266
9
5
14
0.1634
19
29.30
22
0.0273
9
0.0047
30
0.0257
29
20.25
19
0.0106
0.0054
6.00
27
0.0254
29
20.25
20
0.0110
12
6.91
12
0.0098
24
20.92
8
25.46
2
6.91
11
0.0099
24
21.43
9
12.12
1
6
12.73
2
0.1089
26
8.66
3
2
6
12.73
3
0.1133
27
8.66
0.0126
1
19
29.00
25
0.0500
8
0.0167
2
19
29.00
26
0.0461
8
15
1
29
20.07
20
0.0192
6.11
2
29
20.07
19
0.0190
12.93
1
24
20.84
9
3
2
24
21.27
8
0.0236
1
26
8.62
0.0288
2
27
8.62
16
1
8
6.06
2
8
12.73
1
3
2
2
2
1
Sub. #
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Table 70: Tabulation of median search template sizes for Class B. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between
the two submissions each participant could make. Size values indicate the median kilobytes used to store a search template on disk for a single subject, where 1 kB is equal to 1 024 bytes. The
FNIR column shows FNIR for each submission at FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. The number to the left of a value
provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Ten-Finger Plain-to-Plain
Size
FNIR
Sub. #
167
Ten-Finger Rolled-to-Rolled
Size
FNIR
9
20.80
30
NA
14
40.13
16
0.0094
2
9
20.80
24
0.0711
14
40.13
15
0.0085
1
19
33.58
6
0.0015
20
63.49
4
0.0015
2
18
33.00
2
0.0011
25
65.19
7
0.0018
1
1
12.07
14
0.0088
1
19.69
18
0.0106
2
28
66.94
13
0.0048
28
108.80
12
0.0050
1
5
19.35
25
0.0734
3
27.48
25
0.0536
2
5
19.35
25
0.0734
3
27.48
25
0.0536
1
14
26.87
23
0.0368
10
30.12
24
0.0447
2
14
26.87
20
0.0276
10
30.12
21
0.0333
1
16
30.00
20
0.0276
8
30.00
20
0.0201
2
16
30.00
19
0.0275
8
30.00
19
0.0199
1
4
17.86
4
0.0013
2
23.92
1
0.0013
2
11
23.17
1
0.0010
7
28.90
2
0.0014
1
20
38.52
12
0.0047
21
63.55
13
0.0051
2
20
38.52
10
0.0027
21
63.55
9
0.0033
1
2
15.77
16
0.0102
12
32.65
17
0.0097
2
2
15.77
15
0.0095
12
32.65
14
0.0083
1
5
19.35
28
0.0934
3
27.48
28
0.0783
2
5
19.35
27
0.0826
3
27.48
27
0.0716
1
20
38.52
9
0.0025
21
63.55
11
0.0034
2
20
38.52
10
0.0027
21
63.55
9
0.0033
1
29
83.49
2
0.0011
29
135.01
5
0.0017
2
29
83.49
4
0.0013
29
135.01
2
0.0014
1
24
53.59
22
0.0311
26
78.67
29
0.0860
2
24
53.59
29
0.1680
26
78.67
30
0.2462
1
26
63.23
18
0.0163
18
54.73
23
0.0358
2
26
63.23
17
0.0155
18
54.73
22
0.0351
1
12
25.68
7
0.0024
16
48.85
5
0.0017
2
12
25.68
7
0.0024
16
48.85
8
0.0019
1
Table 71: Tabulation of median search template sizes for Class C. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is
an identifier used to differentiate between the two submissions each participant could make. Size values indicate the median kilobytes used to store a
search template on disk for a single subject, where 1 kB is equal to 1 024 bytes. The FNIR column shows FNIR for each submission at FPIR = 10−3 . NA
indicates that the operations required to produce the value could not be performed. The number to the left of a value provides the value’s column-wise
ranking, with the best performance shaded in green and the worst in pink.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
168
H
FPVTE – F INGERPRINT M ATCHING
Template Creation Times
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
K
L
M
O
P
Q
S
T
U
V
Enrollment
Mean
Sub. #
169
Search
Median
Mean
Median
FNIR @ FPIR = 10−3
1
21
0.45
19
0.41
19
0.41
19
0.39
30
0.1335
2
21
0.45
19
0.41
19
0.41
19
0.39
31
0.1337
1
35
1.79
34
1.46
35
1.57
34
1.43
1
0.0197
0.0197
2
34
1.79
33
1.46
34
1.55
33
1.42
1
1
11
0.37
15
0.36
15
0.35
15
0.35
16
0.0745
2
11
0.37
15
0.36
15
0.35
15
0.35
15
0.0723
1
31
1.21
31
1.13
29
1.11
29
1.07
25
0.1111
2
31
1.21
31
1.13
29
1.11
29
1.07
22
0.1082
1
19
0.43
21
0.41
21
0.57
23
0.55
24
0.1089
0.1086
2
19
0.43
21
0.41
21
0.57
23
0.55
23
1
13
0.37
7
0.31
11
0.33
7
0.30
32
0.1576
2
13
0.37
7
0.31
11
0.33
7
0.30
33
0.1607
1
33
1.49
35
1.47
33
1.48
35
1.46
7
0.0257
2
36
2.29
36
2.20
36
2.19
36
2.15
8
0.0278
1
7
0.35
13
0.34
9
0.33
13
0.32
18
0.0786
2
7
0.35
13
0.34
9
0.33
13
0.32
14
0.0712
1
27
1.09
27
1.07
27
1.06
27
1.05
21
0.0883
2
27
1.09
27
1.07
27
1.06
27
1.05
20
0.0875
1
1
0.07
1
0.06
1
0.07
1
0.05
11
0.0625
2
4
0.19
4
0.16
4
0.17
4
0.15
9
0.0351
1
29
1.21
29
1.13
31
1.12
31
1.07
35
0.2995
2
29
1.21
29
1.13
31
1.12
31
1.07
34
0.2921
1
5
0.29
5
0.28
5
0.28
5
0.27
19
0.0818
2
5
0.29
5
0.28
5
0.28
5
0.27
17
0.0766
1
17
0.42
17
0.38
17
0.38
17
0.36
29
0.1308
2
17
0.42
17
0.38
17
0.38
17
0.36
28
0.1272
1
23
0.64
23
0.56
23
0.57
21
0.54
3
0.0222
2
23
0.64
23
0.56
23
0.57
21
0.54
4
0.0226
0.0571
1
25
0.91
25
0.89
25
0.90
25
0.88
10
2
25
0.91
25
0.89
25
0.90
25
0.88
12
0.0650
1
9
0.35
9
0.32
7
0.33
9
0.31
36
NA
2
9
0.35
9
0.32
7
0.33
9
0.31
13
0.0685
1
2
0.17
2
0.16
2
0.16
2
0.15
27
0.1218
2
2
0.17
2
0.16
2
0.16
2
0.15
26
0.1178
0.32
6
0.0253
0.32
5
0.0252
1
15
2
15
0.37
11
0.37
11
0.33
13
0.33
13
0.34
11
0.34
11
Table 72: Tabulation of enrollment time results for Class A — Left Index. Letter refers to the participant’s letter code found on the footer of this page. Sub.
# is an identifier used to differentiate between the two submissions each participant could make. Enrollment shows the time used to create a fingerprint
template to be used in an enrollment set. Search shows the time used to create a search template to be used for a query. All values are reported in
seconds, but were originally recorded to microsecond precision. The number to the left of a value provides the value’s column-wise ranking, with the
best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 7 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
170
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
K
L
M
O
P
Q
S
T
U
V
Enrollment
Mean
Sub. #
Search
Median
Mean
Median
FNIR @ FPIR = 10−3
1
21
0.45
19
0.41
19
0.41
19
0.39
30
0.1132
2
21
0.45
19
0.41
19
0.41
19
0.39
29
0.1124
1
34
1.78
33
1.45
35
1.58
34
1.43
1
0.0190
0.0190
2
35
1.80
34
1.46
34
1.57
33
1.43
1
1
13
0.37
15
0.37
15
0.37
15
0.36
15
0.0630
2
13
0.37
15
0.37
15
0.37
15
0.36
14
0.0624
1
29
1.23
29
1.15
31
1.14
29
1.09
25
0.0933
2
29
1.23
29
1.15
31
1.14
29
1.09
22
0.0903
1
19
0.43
21
0.41
21
0.57
23
0.55
24
0.0910
0.0909
2
19
0.43
21
0.41
21
0.57
23
0.55
23
1
15
0.38
7
0.31
9
0.34
7
0.30
32
0.1230
2
15
0.38
7
0.31
9
0.34
7
0.30
33
0.1249
1
33
1.51
35
1.48
33
1.49
35
1.46
5
0.0215
2
36
2.29
36
2.20
36
2.19
36
2.15
3
0.0214
1
7
0.35
13
0.33
11
0.34
13
0.33
20
0.0708
2
7
0.35
13
0.33
11
0.34
13
0.33
16
0.0643
1
27
1.09
27
1.06
27
1.06
27
1.05
18
0.0682
2
27
1.09
27
1.06
27
1.06
27
1.05
19
0.0685
1
1
0.07
1
0.06
1
0.07
1
0.05
12
0.0505
2
4
0.19
4
0.16
4
0.17
4
0.16
9
0.0295
1
31
1.24
31
1.16
29
1.14
31
1.09
36
0.2615
2
31
1.24
31
1.16
29
1.14
31
1.09
35
0.2526
1
5
0.29
5
0.28
5
0.28
5
0.27
21
0.0776
2
5
0.29
5
0.28
5
0.28
5
0.27
17
0.0675
1
17
0.42
17
0.38
17
0.39
17
0.37
31
0.1133
2
17
0.42
17
0.38
17
0.39
17
0.37
28
0.1100
1
23
0.64
23
0.56
23
0.58
21
0.54
6
0.0218
2
23
0.64
23
0.56
23
0.58
21
0.54
3
0.0214
0.0442
1
25
0.91
25
0.89
25
0.89
25
0.88
10
2
25
0.91
25
0.89
25
0.89
25
0.88
11
0.0503
1
9
0.35
9
0.32
7
0.33
9
0.31
34
0.1929
2
9
0.35
9
0.32
7
0.33
9
0.31
13
0.0562
1
2
0.17
2
0.16
2
0.16
2
0.15
26
0.0996
2
2
0.17
2
0.16
2
0.16
2
0.15
27
0.1007
0.32
8
0.0223
0.32
7
0.0222
1
11
2
11
0.37
11
0.37
11
0.33
13
0.33
13
0.34
11
0.34
11
Table 73: Tabulation of enrollment time results for Class A — Right Index. Letter refers to the participant’s letter code found on the footer of this page. Sub.
# is an identifier used to differentiate between the two submissions each participant could make. Enrollment shows the time used to create a fingerprint
template to be used in an enrollment set. Search shows the time used to create a search template to be used for a query. All values are reported in
seconds, but were originally recorded to microsecond precision. The number to the left of a value provides the value’s column-wise ranking, with the
best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 8 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
K
L
M
O
P
Q
S
T
U
V
Enrollment
Mean
Sub. #
171
Search
Median
Mean
Median
FNIR @ FPIR = 10−3
1
21
0.88
21
0.80
19
0.80
19
0.78
26
0.0368
2
21
0.88
21
0.80
19
0.80
19
0.78
28
0.0374
1
35
3.57
34
2.91
34
3.10
33
2.84
4
0.0030
0.0030
2
34
3.55
33
2.89
35
3.12
34
2.84
4
1
15
0.74
15
0.73
15
0.70
15
0.70
15
0.0207
2
15
0.74
15
0.73
15
0.70
15
0.70
14
0.0202
1
29
2.42
29
2.26
29
2.23
29
2.15
29
0.0386
2
29
2.42
29
2.26
29
2.23
29
2.15
30
0.0412
1
19
0.84
19
0.80
21
1.13
23
1.09
31
0.0515
0.0311
2
19
0.84
19
0.80
21
1.13
23
1.09
20
1
13
0.74
7
0.62
9
0.66
7
0.60
33
0.0686
2
13
0.74
7
0.62
9
0.66
7
0.60
32
0.0684
1
33
2.98
35
2.96
33
2.96
35
2.93
8
0.0058
2
36
4.59
36
4.40
36
4.37
36
4.30
4
0.0030
1
7
0.69
13
0.67
11
0.66
13
0.65
10
0.0143
2
7
0.69
13
0.67
11
0.66
13
0.65
10
0.0143
1
27
2.17
27
2.13
27
2.11
27
2.09
24
0.0360
2
27
2.17
27
2.13
27
2.11
27
2.09
19
0.0286
1
1
0.12
1
0.11
1
0.11
1
0.11
12
0.0146
2
4
0.36
4
0.32
4
0.33
4
0.31
9
0.0072
1
31
2.42
31
2.27
31
2.25
31
2.16
35
NA
2
31
2.42
31
2.27
31
2.25
31
2.16
34
NA
1
5
0.57
5
0.55
5
0.55
5
0.53
17
0.0229
2
5
0.57
5
0.55
5
0.55
5
0.53
16
0.0214
1
17
0.83
17
0.76
17
0.77
17
0.72
27
0.0370
2
17
0.83
17
0.76
17
0.77
17
0.72
21
0.0333
1
23
1.27
23
1.10
23
1.13
21
1.08
1
0.0027
2
23
1.27
23
1.10
23
1.13
21
1.08
1
0.0027
0.0281
1
25
1.81
25
1.78
25
1.77
25
1.75
18
2
25
1.81
25
1.78
25
1.77
25
1.75
13
0.0195
1
9
0.70
9
0.64
7
0.65
9
0.62
36
NA
2
9
0.70
9
0.64
7
0.65
9
0.62
25
0.0366
1
2
0.32
2
0.31
2
0.30
2
0.30
22
0.0336
2
2
0.32
2
0.31
2
0.30
2
0.30
23
0.0358
0.64
7
0.0034
0.64
3
0.0028
1
11
2
11
0.73
11
0.73
11
0.65
13
0.65
13
0.68
11
0.68
11
Table 74: Tabulation of enrollment time results for Class A — Left and Right Index. Letter refers to the participant’s letter code found on the footer of this
page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Enrollment shows the time used to create a
fingerprint template to be used in an enrollment set. Search shows the time used to create a search template to be used for a query. All values are reported
in seconds, but were originally recorded to microsecond precision. The number to the left of a value provides the value’s column-wise ranking, with the
best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 9 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
172
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Search
Mean
Sub. #
Median
FNIR @ FPIR = 10−3
2.12
22
0.0654
2.13
12
2.12
21
0.0647
14
2.16
14
2.15
6
0.0163
2
24
4.18
24
4.17
5
0.0142
1
11
1.95
11
1.95
13
0.0259
2
15
2.21
15
2.21
7
0.0187
0.1684
1
12
2.13
12
2
12
1
1
28
6.94
26
6.73
29
2
28
6.94
26
6.73
28
0.1681
1
18
2.81
18
2.83
18
0.0371
2
18
2.81
18
2.83
17
0.0325
1
9
1.68
9
1.67
23
0.0998
2
9
1.68
9
1.67
24
0.1008
1
25
6.50
25
6.50
4
0.0116
2
30
7.68
30
7.67
1
0.0094
1
7
1.22
3
1.21
15
0.0287
2
7
1.22
3
1.21
10
0.0236
1
2
1.13
2
1.11
16
0.0288
2
1
0.33
1
0.33
14
0.0276
1
26
6.94
28
6.73
30
0.1736
2
26
6.94
28
6.73
27
0.1634
1
5
1.22
5
1.21
12
0.0257
2
5
1.22
5
1.21
11
0.0254
1
20
3.41
20
3.41
2
0.0098
2
20
3.41
20
3.41
3
0.0099
1
22
3.97
22
3.96
25
0.1089
0.1133
2
22
3.97
22
3.96
26
1
17
2.46
17
2.42
20
0.0500
2
16
2.45
16
2.41
19
0.0461
1
3
1.21
7
1.22
9
0.0192
2
3
1.21
7
1.22
8
0.0190
Table 75: Tabulation of enrollment time results for Class B — Left Slap. Letter refers to the participant’s letter code found on the footer of this page. Sub. #
is an identifier used to differentiate between the two submissions each participant could make. Search shows the time used to create a search template to
be used for a query. All values are reported in seconds, but were originally recorded to microsecond precision. The number to the left of a value provides
the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 10 are
reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Search
Mean
Sub. #
Median
173
FNIR @ FPIR = 10−3
2.09
24
0.0403
2.10
12
2.09
23
0.0392
14
2.15
14
2.14
6
0.0072
2
24
4.15
24
4.13
2
0.0052
1
11
1.92
11
1.91
13
0.0151
2
15
2.18
15
2.18
7
0.0083
0.1222
1
12
2.10
12
2
12
1
1
26
6.86
26
6.68
29
2
26
6.86
26
6.68
28
0.1220
1
18
2.76
18
2.78
18
0.0212
2
18
2.76
18
2.78
16
0.0198
1
9
1.64
9
1.63
25
0.0641
2
9
1.64
9
1.63
26
0.0647
1
25
6.47
25
6.47
5
0.0058
2
30
7.65
30
7.65
1
0.0045
1
5
1.21
5
1.20
14
0.0156
2
5
1.21
5
1.20
10
0.0126
1
2
1.12
2
1.11
15
0.0167
2
1
0.33
1
0.33
17
0.0202
1
28
6.88
28
6.69
30
0.1259
2
28
6.88
28
6.69
27
0.1155
1
7
1.21
7
1.20
12
0.0142
2
7
1.21
7
1.20
11
0.0132
1
20
3.41
20
3.41
3
0.0057
2
20
3.41
20
3.41
3
0.0057
1
22
3.89
22
3.88
21
0.0369
0.0381
2
22
3.89
22
3.88
22
1
16
2.43
16
2.39
19
0.0266
2
17
2.43
17
2.39
20
0.0273
1
3
1.19
3
1.19
8
0.0106
2
3
1.19
3
1.19
9
0.0110
Table 76: Tabulation of enrollment time results for Class B — Right Slap. Letter refers to the participant’s letter code found on the footer of this page.
Sub. # is an identifier used to differentiate between the two submissions each participant could make. Search shows the time used to create a search
template to be used for a query. All values are reported in seconds, but were originally recorded to microsecond precision. The number to the left of a
value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from
Table 11 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
174
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Search
Mean
Sub. #
Median
FNIR @ FPIR = 10−3
4.20
30
NA
4.22
12
4.20
29
NA
14
4.29
14
4.28
6
0.0031
2
24
8.31
24
8.28
5
0.0024
1
11
3.86
11
3.85
15
0.0063
2
15
4.39
15
4.38
10
0.0049
0.0910
1
12
4.22
12
2
12
1
1
26
13.78
26
13.49
28
2
26
13.78
26
13.49
26
0.0901
1
18
5.59
18
5.62
18
0.0106
2
18
5.59
18
5.62
17
0.0084
1
9
3.31
9
3.31
23
0.0349
2
9
3.31
9
3.31
24
0.0361
1
25
12.94
25
12.96
3
0.0022
2
30
15.30
30
15.33
1
0.0015
1
7
2.42
5
2.40
16
0.0068
2
7
2.42
5
2.40
9
0.0047
1
2
2.24
2
2.23
12
0.0054
2
1
0.66
1
0.66
14
0.0062
1
28
13.79
28
13.53
27
0.0904
2
28
13.79
28
13.53
25
0.0882
1
5
2.42
3
2.40
13
0.0057
2
5
2.42
3
2.40
11
0.0051
1
20
6.84
20
6.84
2
0.0021
2
20
6.84
20
6.84
3
0.0022
1
22
7.82
22
7.83
21
0.0160
0.0190
2
22
7.82
22
7.83
22
1
17
4.91
16
4.82
20
0.0139
2
16
4.89
17
4.83
19
0.0124
1
3
2.40
7
2.40
7
0.0036
2
3
2.40
7
2.40
7
0.0036
Table 77: Tabulation of enrollment time results for Class B — Left and Right Slap. Letter refers to the participant’s letter code found on the footer of this
page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Search shows the time used to create a search
template to be used for a query. All values are reported in seconds, but were originally recorded to microsecond precision. The number to the left of a
value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. For reference, the FNIR values from
Table 12 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Enrollment
Mean
Sub. #
1
11
175
Search
Median
Mean
Median
FNIR @ FPIR = 10−3
5.13
30
NA
5.16
11
5.13
29
NA
17
6.19
17
6.16
6
0.0020
11.47
24
11.90
24
11.86
2
0.0012
13
5.28
13
5.35
13
5.34
16
0.0043
18
5.89
16
5.95
16
5.93
7
0.0024
0.0591
4.92
11
5.06
11
5.16
11
2
11
4.92
11
5.06
11
1
19
5.82
19
5.97
2
24
11.20
24
1
13
5.11
2
18
5.69
1
28
17.69
28
17.86
28
18.36
26
18.07
27
2
28
17.69
28
17.86
28
18.36
26
18.07
27
0.0591
1
14
5.41
14
5.60
18
7.89
18
7.90
18
0.0062
2
14
5.41
14
5.60
18
7.89
18
7.90
14
0.0040
1
9
4.15
9
4.28
9
4.37
9
4.37
23
0.0203
2
9
4.15
9
4.28
9
4.37
9
4.37
24
0.0204
1
25
15.80
25
16.47
25
16.50
25
16.52
2
0.0012
2
30
18.56
30
19.35
30
19.36
30
19.36
1
0.0009
1
3
3.20
3
3.27
7
3.38
3
3.35
17
0.0049
2
3
3.20
3
3.27
7
3.38
3
3.35
11
0.0033
1
2
2.94
2
3.02
2
3.05
2
3.05
10
0.0031
2
1
0.84
1
0.87
1
0.88
1
0.88
11
0.0033
1
26
17.66
26
17.85
26
18.36
28
18.11
26
0.0543
2
26
17.66
26
17.85
26
18.36
28
18.11
25
0.0515
1
7
3.20
5
3.27
5
3.38
5
3.36
15
0.0041
2
7
3.20
5
3.27
5
3.38
5
3.36
13
0.0035
1
20
8.50
20
8.85
20
8.89
20
8.92
2
0.0012
2
20
8.50
20
8.85
20
8.89
20
8.92
2
0.0012
1
22
9.80
22
10.19
22
10.24
22
10.31
20
0.0108
0.0136
2
22
9.80
22
10.19
22
10.24
22
10.31
21
1
17
5.66
17
5.73
15
5.90
15
5.82
19
0.0099
2
16
5.56
16
5.66
14
5.80
14
5.73
22
0.0141
1
5
3.20
7
3.30
3
3.35
7
3.36
9
0.0027
2
5
3.20
7
3.30
3
3.35
7
3.36
7
0.0024
Table 78: Tabulation of enrollment time results for Class B — Identification Flats. Letter refers to the participant’s letter code found on the footer of this
page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Enrollment shows the time used to create a
fingerprint template to be used in an enrollment set. Search shows the time used to create a search template to be used for a query. All values are reported
in seconds, but were originally recorded to microsecond precision. The number to the left of a value provides the value’s column-wise ranking, with the
best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 13 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
176
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Enrollment
Mean
Sub. #
1
15
Search
Median
5.93
13
2
15
1
19
2
Mean
Median
FNIR @ FPIR = 10−3
5.80
30
NA
5.84
13
5.80
24
0.0711
7.12
17
7.10
6
0.0015
24
13.69
24
13.66
2
0.0011
5.21
11
5.20
11
5.20
14
0.0088
5.22
12
5.21
12
5.20
13
0.0048
0.0734
5.89
13
5.84
13
5.93
13
5.89
13
7.10
19
7.09
17
24
13.55
24
13.52
1
11
5.19
11
2
12
5.20
12
1
27
15.96
25
15.88
27
15.96
27
15.87
25
2
27
15.96
25
15.88
27
15.96
27
15.87
25
0.0734
1
13
5.93
15
5.91
18
8.35
18
8.32
23
0.0368
2
13
5.93
15
5.91
18
8.35
18
8.32
20
0.0276
1
9
5.18
9
5.21
9
5.16
9
5.16
20
0.0276
2
9
5.18
9
5.21
9
5.16
9
5.16
19
0.0275
1
29
16.65
29
16.67
29
16.71
29
16.67
4
0.0013
2
30
17.15
30
17.07
30
17.04
30
16.99
1
0.0010
1
3
3.61
3
3.58
3
3.62
5
3.62
12
0.0047
2
6
3.63
6
3.60
6
3.64
6
3.62
10
0.0027
1
1
3.56
1
3.52
1
3.51
1
3.48
16
0.0102
2
1
3.56
1
3.52
1
3.51
1
3.48
15
0.0095
1
25
15.95
27
15.89
25
15.91
25
15.83
28
0.0934
2
25
15.95
27
15.89
25
15.91
25
15.83
27
0.0826
1
4
3.63
4
3.60
4
3.63
3
3.61
9
0.0025
2
4
3.63
4
3.60
4
3.63
3
3.61
10
0.0027
1
20
9.10
20
9.08
20
9.14
20
9.11
2
0.0011
2
20
9.10
20
9.08
20
9.14
20
9.11
4
0.0013
1
22
10.86
22
10.87
22
10.91
22
10.87
22
0.0311
0.1680
2
22
10.86
22
10.87
22
10.91
22
10.87
29
1
17
6.34
17
6.09
15
6.13
15
5.91
18
0.0163
2
17
6.34
17
6.09
15
6.13
15
5.91
17
0.0155
1
7
3.68
7
3.67
7
3.69
7
3.66
7
0.0024
2
7
3.68
7
3.67
7
3.69
7
3.66
7
0.0024
Table 79: Tabulation of enrollment time results for Class C — Ten-Finger Plain-to-Plain. Letter refers to the participant’s letter code found on the footer
of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Enrollment shows the time used to
create a fingerprint template to be used in an enrollment set. Search shows the time used to create a search template to be used for a query. All values are
reported in seconds, but were originally recorded to microsecond precision. The number to the left of a value provides the value’s column-wise ranking,
with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 14 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
C
D
E
F
G
H
I
J
L
M
O
Q
S
U
V
Enrollment
Mean
Sub. #
1
13
177
Search
Median
Mean
Median
FNIR @ FPIR = 10−3
10.69
16
0.0094
10.79
13
10.69
15
0.0085
23
17.44
21
17.39
4
0.0015
30.70
30
30.43
30
30.47
7
0.0018
12
9.94
12
9.90
12
9.92
18
0.0106
11
9.94
11
9.89
11
9.91
12
0.0050
0.0536
11.18
13
11.00
13
10.79
13
2
13
11.18
13
11.00
13
1
21
17.37
21
17.27
2
30
30.78
30
1
12
9.93
2
11
9.92
1
26
21.06
28
20.95
28
21.07
28
20.92
25
2
26
21.06
28
20.95
28
21.07
28
20.92
25
0.0536
1
15
11.21
15
11.19
19
15.55
19
15.50
24
0.0447
2
15
11.21
15
11.19
19
15.55
19
15.50
21
0.0333
1
17
12.36
17
12.36
15
12.08
15
12.17
20
0.0201
2
17
12.36
17
12.36
15
12.08
15
12.17
19
0.0199
1
25
20.25
25
20.25
25
20.21
25
20.23
1
0.0013
2
24
18.75
24
18.76
24
18.72
24
18.73
2
0.0014
1
7
6.78
5
6.67
5
6.78
5
6.69
13
0.0051
2
8
6.80
8
6.71
8
6.82
8
6.74
9
0.0033
1
3
4.43
3
4.44
3
4.36
3
4.36
17
0.0097
2
3
4.43
3
4.44
3
4.36
3
4.36
14
0.0083
1
28
21.08
26
20.92
26
21.02
26
20.89
28
0.0783
2
28
21.08
26
20.92
26
21.02
26
20.89
27
0.0716
1
5
6.77
6
6.67
6
6.80
6
6.71
11
0.0034
2
5
6.77
6
6.67
6
6.80
6
6.71
9
0.0033
1
19
12.94
19
13.03
17
12.95
17
13.06
5
0.0017
2
19
12.94
19
13.03
17
12.95
17
13.06
2
0.0014
1
22
17.54
22
17.67
21
17.43
22
17.57
29
0.0860
0.2462
2
22
17.54
22
17.67
21
17.43
22
17.57
30
1
1
2.95
1
2.88
1
2.94
1
2.87
23
0.0358
2
1
2.95
1
2.88
1
2.94
1
2.87
22
0.0351
1
9
8.60
9
8.62
9
8.48
9
8.50
5
0.0017
2
9
8.60
9
8.62
9
8.48
9
8.50
8
0.0019
Table 80: Tabulation of enrollment time results for Class C — Ten-Finger Rolled-to-Rolled. Letter refers to the participant’s letter code found on the footer
of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Enrollment shows the time used to
create a fingerprint template to be used in an enrollment set. Search shows the time used to create a search template to be used for a query. All values are
reported in seconds, but were originally recorded to microsecond precision. The number to the left of a value provides the value’s column-wise ranking,
with the best performance shaded in green and the worst in pink. For reference, the FNIR values from Table 15 are reprinted to the right of this table.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
178
I
FPVTE – F INGERPRINT M ATCHING
Ranked Results
In order to reduce the number of tables in Section 10 of the main body of the report, this appendix contains the tables
showing ranked results for all three classes. The tables from the main body of the document are repeated in this appendix
so there is a complete set of tables here for the reader to analyze.
Class A results are in Tables 81 through 83, Class B results are in Tables 84 through 87, and Class C results are in Tables 88
through 89.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
≥ 20 seconds
< 20 seconds
Participant
Letter
Sub. #
Identification
Mean
Median
FNIR
179
Search Enrollment
Mean
Median
RAM
D
1
1
0.0197
20
7.52
20
7.32
30
1.57
29
1.43
19
0.61
Q
1
2
0.0222
29
15.90
28
15.70
20
0.57
19
0.54
13
0.34
Q
2
3
0.0226
23
10.42
24
10.43
20
0.57
19
0.54
14
0.34
V
1
4
0.0253
19
6.22
19
6.01
12
0.34
10
0.32
17
0.39
I
1
5
0.0257
22
10.36
22
9.94
29
1.48
30
1.46
20
0.70
L
2
6
0.0351
12
3.34
12
3.32
3
0.17
3
0.15
28
2.32
S
1
7
0.0571
28
15.37
29
16.86
22
0.90
22
0.88
21
0.72
L
1
8
0.0625
2
0.30
2
0.29
1
0.07
1
0.05
26
1.14
T
2
9
0.0685
18
5.96
18
5.97
6
0.33
8
0.31
1
0.01
J
2
10
0.0712
7
1.34
7
1.25
8
0.33
11
0.32
9
0.32
E
2
11
0.0723
30
16.71
30
16.90
13
0.35
13
0.35
24
1.11
E
1
12
0.0745
4
0.57
3
0.35
13
0.35
13
0.35
25
1.12
O
2
13
0.0766
9
1.63
9
1.56
4
0.28
4
0.27
15
0.35
J
1
14
0.0786
3
0.56
4
0.54
8
0.33
11
0.32
9
0.32
O
1
15
0.0818
5
0.64
5
0.62
4
0.28
4
0.27
15
0.35
K
2
16
0.0875
24
10.47
23
10.32
23
1.06
23
1.05
30
3.87
K
1
17
0.0883
25
10.48
25
10.44
23
1.06
23
1.05
29
3.87
F
2
18
0.1082
15
3.78
14
3.56
25
1.11
25
1.07
5
0.18
G
1
19
0.1089
21
9.52
21
9.74
19
0.57
21
0.55
18
0.57
F
1
20
0.1111
11
2.52
11
2.49
25
1.11
25
1.07
6
0.18
U
1
21
0.1218
26
14.72
27
14.42
2
0.16
2
0.15
27
1.70
P
2
22
0.1272
13
3.43
13
3.33
15
0.38
15
0.36
22
0.78
P
1
23
0.1308
8
1.38
8
1.32
15
0.38
15
0.36
23
0.78
C
1
24
0.1335
1
0.29
1
0.26
17
0.41
17
0.39
7
0.18
C
2
25
0.1337
6
0.87
6
0.76
17
0.41
17
0.39
8
0.18
H
1
26
0.1576
14
3.65
15
3.61
10
0.33
6
0.30
11
0.33
H
2
27
0.1607
17
4.26
17
4.13
10
0.33
6
0.30
11
0.33
M
2
28
0.2921
27
15.19
26
10.70
27
1.12
27
1.07
4
0.14
M
1
29
0.2995
10
2.07
10
1.84
27
1.12
27
1.07
3
0.14
T
1
30
NA
16
4.18
16
3.99
6
0.33
8
0.31
2
0.01
D
2
1
0.0197
2
42.64
3
41.84
5
1.55
5
1.42
3
0.61
V
2
2
0.0252
6
66.23
6
65.35
2
0.34
2
0.32
1
0.39
I
2
3
0.0278
3
43.24
2
40.99
6
2.19
6
2.15
5
1.04
S
2
4
0.0650
4
45.59
4
44.66
4
0.90
4
0.88
4
0.72
G
2
5
0.1086
5
54.24
5
54.03
3
0.57
3
0.55
2
0.57
U
2
6
0.1178
1
25.66
1
24.16
1
0.16
1
0.15
6
1.42
Table 81: Tabulation of ranked results for Class A — Left Index. Submissions were split into two groups. The first group includes submissions that
performed searches on average in less than 20 seconds, and the second includes those that took, on average, 20 seconds or longer. Letter refers to the
participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could
make. The FNIR column was computed at the score threshold that gave FPIR = 10−3 . NA indicates that the operations required to produce the value
could not be performed. The Identification column shows the time used to perform a search over an enrollment set of 100 000, as seen in Table 17. The
Search Enrollment column shows the time used to create a search template to be used for a query, as seen in Table 72. Identification and Search Enrollment
durations are reported in seconds, but were originally recorded to microsecond precision. RAM refers to the sum of the resident set sizes of the stage
one identification processes over all compute nodes after returning from the identification stage one initialization method, as seen in Table 60. RAM is
reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s column-wise ranking, with the
best performance shaded in green and the worst in pink. The table is sorted on the FNIR column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
180
FPVTE – F INGERPRINT M ATCHING
≥ 20 seconds
< 20 seconds
Participant
Letter
Sub. #
Identification
Mean
Median
FNIR
Search Enrollment
Mean
Median
RAM
D
1
1
0.0190
20
7.68
20
7.46
30
1.58
29
1.43
19
0.61
Q
2
2
0.0214
22
10.15
23
9.98
20
0.58
19
0.54
11
0.33
I
1
3
0.0215
23
10.19
22
9.83
29
1.49
30
1.46
20
0.69
Q
1
4
0.0218
28
14.44
28
14.24
20
0.58
19
0.54
11
0.33
V
1
5
0.0223
19
5.95
18
5.60
12
0.34
10
0.32
17
0.39
L
2
6
0.0295
12
3.26
13
3.26
3
0.17
3
0.16
28
2.32
S
1
7
0.0442
29
15.08
30
16.55
22
0.89
22
0.88
21
0.71
L
1
8
0.0505
2
0.29
2
0.26
1
0.07
1
0.05
26
1.14
T
2
9
0.0562
18
5.64
19
5.87
6
0.33
8
0.31
1
0.01
E
2
10
0.0624
30
15.89
29
16.27
13
0.37
13
0.36
25
1.07
E
1
11
0.0630
3
0.45
3
0.32
13
0.37
13
0.36
24
1.07
J
2
12
0.0643
7
1.25
7
1.13
10
0.34
11
0.33
9
0.31
O
2
13
0.0675
9
1.48
9
1.36
4
0.28
4
0.27
16
0.34
K
1
14
0.0682
25
10.42
26
10.42
23
1.06
23
1.05
30
3.87
K
2
15
0.0685
24
10.30
25
10.27
23
1.06
23
1.05
29
3.87
J
1
16
0.0708
4
0.53
4
0.51
10
0.34
11
0.33
10
0.31
O
1
17
0.0776
5
0.59
5
0.56
4
0.28
4
0.27
15
0.34
F
2
18
0.0903
15
3.71
15
3.60
27
1.14
25
1.09
5
0.17
G
1
19
0.0910
21
10.12
24
10.10
19
0.57
21
0.55
18
0.57
F
1
20
0.0933
11
2.52
11
2.38
27
1.14
25
1.09
6
0.17
U
1
21
0.0996
27
12.02
27
10.76
2
0.16
2
0.15
27
1.56
P
2
22
0.1100
13
3.49
12
3.07
15
0.39
15
0.37
23
0.76
C
2
23
0.1124
6
0.76
6
0.69
17
0.41
17
0.39
8
0.18
C
1
24
0.1132
1
0.25
1
0.24
17
0.41
17
0.39
7
0.17
P
1
25
0.1133
8
1.38
8
1.24
15
0.39
15
0.37
22
0.76
H
1
26
0.1230
14
3.58
14
3.56
8
0.34
6
0.30
14
0.33
H
2
27
0.1249
17
4.13
17
4.05
8
0.34
6
0.30
13
0.33
T
1
28
0.1929
16
3.82
16
3.73
6
0.33
8
0.31
1
0.01
M
2
29
0.2526
26
11.31
21
9.39
25
1.14
27
1.09
3
0.14
M
1
30
0.2615
10
1.75
10
1.78
25
1.14
27
1.09
3
0.14
D
2
1
0.0190
2
29.26
2
28.64
5
1.57
5
1.43
3
0.61
I
2
2
0.0214
3
41.60
3
40.35
6
2.19
6
2.15
5
1.01
V
2
3
0.0222
6
64.47
6
61.24
2
0.34
2
0.32
1
0.39
S
2
4
0.0503
4
43.56
4
46.05
4
0.89
4
0.88
4
0.71
G
2
5
0.0909
5
57.88
5
59.55
3
0.57
3
0.55
2
0.57
U
2
6
0.1007
1
22.39
1
20.10
1
0.16
1
0.15
6
1.30
Table 82: Tabulation of ranked results for Class A — Right Index. Submissions were split into two groups. The first group includes submissions that
performed searches on average in less than 20 seconds, and the second includes those that took, on average, 20 seconds or longer. Letter refers to the
participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could
make. The FNIR column was computed at the score threshold that gave FPIR = 10−3 . The Identification column shows the time used to perform a search
over an enrollment set of 100 000, as seen in Table 17. The Search Enrollment column shows the time used to create a search template to be used for a
query, as seen in Table 73. Identification and Search Enrollment durations are reported in seconds, but were originally recorded to microsecond precision.
RAM refers to the sum of the resident set sizes of the stage one identification processes over all compute nodes after returning from the identification
stage one initialization method, as seen in Table 61. RAM is reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of
a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. The table is sorted on the FNIR
column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
≥ 20 seconds
< 20 seconds
Participant
Identification
Mean
Median
FNIR
181
Search Enrollment
Mean
Median
RAM
Letter
Sub. #
V
1
1
0.0034
5
9.50
5
9.29
4
0.68
3
0.64
5
11.77
I
1
2
0.0058
9
18.92
8
17.87
9
2.96
9
2.93
6
15.83
J
1
3
0.0143
6
14.00
6
13.64
3
0.66
4
0.65
3
8.30
L
1
4
0.0146
1
2.20
2
2.19
1
0.11
1
0.11
8
18.76
O
1
5
0.0229
7
15.34
7
14.46
2
0.55
2
0.53
4
9.05
K
1
6
0.0360
8
18.32
9
18.01
8
2.11
8
2.09
9
61.81
C
1
7
0.0368
2
2.39
1
2.08
5
0.80
5
0.78
2
4.87
C
2
8
0.0374
4
6.34
4
6.35
5
0.80
5
0.78
1
4.87
G
1
9
0.0515
3
6.27
3
5.22
7
1.13
7
1.09
7
16.38
Q
1
1
0.0027
21
213.08
21
212.69
16
1.13
15
1.08
10
9.14
Q
2
1
0.0027
19
163.65
19
161.02
16
1.13
15
1.08
9
9.14
V
2
3
0.0028
18
133.45
18
127.65
10
0.68
9
0.64
13
11.77
I
2
4
0.0030
25
385.14
25
338.88
27
4.37
27
4.30
21
30.83
D
2
4
0.0030
23
234.52
23
237.43
26
3.12
26
2.84
17
18.58
D
1
4
0.0030
16
73.01
15
70.99
25
3.10
25
2.84
17
18.58
L
2
7
0.0072
3
23.54
3
23.42
3
0.33
3
0.31
26
53.98
J
2
8
0.0143
7
36.19
7
33.35
9
0.66
10
0.65
7
8.30
S
2
9
0.0195
26
429.02
26
495.50
18
1.77
18
1.75
14
14.82
E
2
10
0.0202
27
500.30
27
518.11
11
0.70
11
0.70
23
33.41
E
1
11
0.0207
2
22.27
1
16.35
11
0.70
11
0.70
22
33.40
O
2
12
0.0214
10
45.52
10
43.53
4
0.55
4
0.53
8
9.05
0.0281
1
14.82
S
1
13
20.11
2
23.00
18
1.77
18
1.75
14
K
2
14
0.0286
6
32.68
6
32.93
20
2.11
20
2.09
27
61.81
G
2
15
0.0311
22
227.27
22
221.16
15
1.13
17
1.09
16
16.38
P
2
16
0.0333
17
114.37
17
101.16
13
0.77
13
0.72
19
21.41
U
1
17
0.0336
11
47.60
11
45.51
1
0.30
1
0.30
25
44.63
U
2
18
0.0358
24
252.04
24
240.40
1
0.30
1
0.30
24
37.35
T
2
19
0.0366
9
38.91
9
37.23
5
0.65
7
0.62
1
0.01
P
1
20
0.0370
14
63.41
14
63.65
13
0.77
13
0.72
20
21.42
F
1
21
0.0386
13
59.30
13
60.66
21
2.23
21
2.15
5
4.86
F
2
22
0.0412
15
71.45
16
73.95
21
2.23
21
2.15
5
4.86
H
2
23
0.0684
12
53.77
12
52.25
7
0.66
5
0.60
11
9.36
H
1
24
0.0686
8
37.65
8
36.71
7
0.66
5
0.60
12
9.36
M
2
25
NA
20
183.20
20
171.24
23
2.25
23
2.16
4
3.76
M
1
26
NA
5
32.59
5
31.44
23
2.25
23
2.16
3
3.75
T
1
27
NA
4
26.96
4
25.68
5
0.65
7
0.62
1
0.01
Table 83: Tabulation of ranked results for Class A — Left and Right Index. Submissions were split into two groups. The first group includes submissions
that performed searches on average in less than 20 seconds, and the second includes those that took, on average, 20 seconds or longer. Letter refers to the
participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could
make. The FNIR column was computed at the score threshold that gave FPIR = 10−3 . NA indicates that the operations required to produce the value
could not be performed. The Identification column shows the time used to perform a search over an enrollment set of 1 600 000, as seen in Table 18. The
Search Enrollment column shows the time used to create a search template to be used for a query, as seen in Table 74. Identification and Search Enrollment
durations are reported in seconds, but were originally recorded to microsecond precision. RAM refers to the sum of the resident set sizes of the stage
one identification processes over all compute nodes after returning from the identification stage one initialization method, as seen in Table 62. RAM is
reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s column-wise ranking, with the
best performance shaded in green and the worst in pink. The table is sorted on the FNIR column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
182
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
Sub. #
Identification
Mean
Median
FNIR
Search Enrollment
Mean
Median
RAM
I
2
1
0.0094
18
51.59
13
38.26
30
7.68
30
7.67
21
108.71
Q
1
2
0.0098
17
51.33
19
53.24
20
3.41
20
3.41
1
7.54
Q
2
3
0.0099
16
51.05
20
54.02
20
3.41
20
3.41
2
7.54
I
1
4
0.0116
23
63.06
21
55.96
25
6.50
25
6.50
5
49.68
D
2
5
0.0142
21
59.65
22
58.10
24
4.18
24
4.17
12
79.43
D
1
6
0.0163
20
53.09
18
52.23
14
2.16
14
2.15
13
79.43
E
2
7
0.0187
12
41.57
10
34.37
15
2.21
15
2.21
28
318.03
V
2
8
0.0190
14
48.01
15
47.97
3
1.21
7
1.22
9
63.53
V
1
9
0.0192
10
36.28
11
36.07
3
1.21
7
1.22
8
63.53
J
2
10
0.0236
27
80.54
27
74.38
7
1.22
3
1.21
19
101.00
O
2
11
0.0254
24
78.80
26
73.15
5
1.22
5
1.21
18
101.00
O
1
12
0.0257
11
38.17
12
37.01
5
1.22
5
1.21
17
101.00
E
1
13
0.0259
1
3.71
1
2.82
11
1.95
11
1.95
14
79.72
L
2
14
0.0276
5
12.52
5
12.37
1
0.33
1
0.33
27
177.49
J
1
15
0.0287
7
26.88
7
25.61
7
1.22
3
1.21
20
101.00
L
1
16
0.0288
3
5.51
3
5.56
2
1.13
2
1.11
22
119.79
G
2
17
0.0325
22
62.98
23
59.49
18
2.81
18
2.83
25
156.12
G
1
18
0.0371
6
19.65
6
16.24
18
2.81
18
2.83
26
156.12
U
2
19
0.0461
29
90.50
30
89.07
16
2.45
16
2.41
30
540.60
U
1
20
0.0500
28
81.85
28
80.03
17
2.46
17
2.42
29
440.67
C
2
21
0.0647
4
6.62
4
6.74
12
2.13
12
2.12
4
46.49
C
1
22
0.0654
2
3.71
2
3.74
12
2.13
12
2.12
3
46.49
H
1
23
0.0998
15
49.13
16
49.75
9
1.68
9
1.67
16
86.46
H
2
24
0.1008
19
52.05
17
51.77
9
1.68
9
1.67
15
86.46
S
1
25
0.1089
25
78.95
24
71.69
22
3.97
22
3.96
24
150.35
S
2
26
0.1133
26
79.01
25
71.77
22
3.97
22
3.96
23
150.35
M
2
27
0.1634
30
90.51
29
87.21
26
6.94
28
6.73
7
57.53
F
2
28
0.1681
13
46.18
14
43.59
28
6.94
26
6.73
10
77.02
F
1
29
0.1684
8
31.92
8
30.09
28
6.94
26
6.73
11
77.02
M
1
30
0.1736
9
32.39
9
31.17
26
6.94
28
6.73
6
57.53
Table 84: Tabulation of ranked results for Class B — Left Slap. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is
an identifier used to differentiate between the two submissions each participant could make. The FNIR column was computed at the score threshold
that gave FPIR = 10−3 . The Identification column shows the time used to perform a search over an enrollment set of 3 000 000, as seen in Table 19. The
Search Enrollment column shows the time used to create a search template to be used for a query, as seen in Table 75. Identification and Search Enrollment
durations are reported in seconds, but were originally recorded to microsecond precision. RAM refers to the sum of the resident set sizes of the stage
one identification processes over all compute nodes after returning from the identification stage one initialization method, as seen in Table 63. RAM is
reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s column-wise ranking, with the
best performance shaded in green and the worst in pink. The table is sorted on the FNIR column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
Sub. #
Identification
Mean
Median
FNIR
183
Search Enrollment
Mean
Median
RAM
I
2
1
0.0045
16
49.18
14
41.13
30
7.65
30
7.65
21
108.71
D
2
2
0.0052
20
58.67
21
57.19
24
4.15
24
4.13
12
79.43
Q
1
3
0.0057
22
62.95
23
64.04
20
3.41
20
3.41
1
7.54
Q
2
3
0.0057
21
61.29
22
61.94
20
3.41
20
3.41
2
7.54
I
1
5
0.0058
23
63.57
20
55.87
25
6.47
25
6.47
5
49.68
D
1
6
0.0072
19
53.78
19
53.32
14
2.15
14
2.14
13
79.43
E
2
7
0.0083
12
40.27
10
34.47
15
2.18
15
2.18
28
318.00
V
1
8
0.0106
10
35.66
11
35.43
3
1.19
3
1.19
8
63.53
V
2
9
0.0110
15
48.47
16
48.51
3
1.19
3
1.19
9
63.53
J
2
10
0.0126
27
77.67
27
74.09
5
1.21
5
1.20
19
101.00
O
2
11
0.0132
26
77.37
26
73.93
7
1.21
7
1.20
20
101.00
O
1
12
0.0142
11
38.08
12
35.61
7
1.21
7
1.20
17
101.00
E
1
13
0.0151
2
4.68
1
2.97
11
1.92
11
1.91
14
79.72
J
1
14
0.0156
7
25.86
7
24.14
5
1.21
5
1.20
18
101.00
L
1
15
0.0167
3
5.49
3
5.47
2
1.12
2
1.11
22
119.78
G
2
16
0.0198
13
41.12
13
37.28
18
2.76
18
2.78
25
156.12
L
2
17
0.0202
6
12.66
6
12.66
1
0.33
1
0.33
27
177.49
G
1
18
0.0212
5
11.80
5
10.13
18
2.76
18
2.78
26
156.12
U
1
19
0.0266
29
89.71
30
89.26
16
2.43
16
2.39
29
440.67
U
2
20
0.0273
28
88.88
28
80.11
17
2.43
17
2.39
30
540.59
S
1
21
0.0369
25
74.29
24
70.54
22
3.89
22
3.88
23
150.35
S
2
22
0.0381
24
73.96
25
70.84
22
3.89
22
3.88
24
150.35
C
2
23
0.0392
4
6.54
4
6.50
12
2.10
12
2.09
3
46.49
C
1
24
0.0403
1
3.70
2
3.67
12
2.10
12
2.09
4
46.49
H
1
25
0.0641
17
50.97
17
50.16
9
1.64
9
1.63
16
86.46
H
2
26
0.0647
18
52.86
18
52.84
9
1.64
9
1.63
15
86.46
M
2
27
0.1155
30
91.42
29
87.41
28
6.88
28
6.69
6
57.53
F
2
28
0.1220
14
46.58
15
45.92
26
6.86
26
6.68
10
77.02
F
1
29
0.1222
8
32.28
8
31.87
26
6.86
26
6.68
11
77.02
M
1
30
0.1259
9
33.43
9
32.03
28
6.88
28
6.69
7
57.53
Table 85: Tabulation of ranked results for Class B — Right Slap. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is
an identifier used to differentiate between the two submissions each participant could make. The FNIR column was computed at the score threshold
that gave FPIR = 10−3 . The Identification column shows the time used to perform a search over an enrollment set of 3 000 000, as seen in Table 19. The
Search Enrollment column shows the time used to create a search template to be used for a query, as seen in Table 76. Identification and Search Enrollment
durations are reported in seconds, but were originally recorded to microsecond precision. RAM refers to the sum of the resident set sizes of the stage
one identification processes over all compute nodes after returning from the identification stage one initialization method, as seen in Table 63. RAM is
reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s column-wise ranking, with the
best performance shaded in green and the worst in pink. The table is sorted on the FNIR column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
184
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
Sub. #
Identification
Mean
Median
FNIR
Search Enrollment
Mean
Median
RAM
I
2
1
0.0015
19
57.94
17
46.13
30
15.30
30
15.33
21
108.71
Q
1
2
0.0021
20
63.99
22
65.02
20
6.84
20
6.84
2
7.54
Q
2
3
0.0022
18
53.88
19
53.40
20
6.84
20
6.84
1
7.54
I
1
3
0.0022
15
47.35
13
37.19
25
12.94
25
12.96
5
49.68
D
2
5
0.0024
24
74.21
21
63.05
24
8.31
24
8.28
13
79.43
D
1
6
0.0031
21
69.35
20
61.24
14
4.29
14
4.28
12
79.43
V
1
7
0.0036
11
39.24
15
38.93
3
2.40
7
2.40
9
63.53
V
2
7
0.0036
17
52.97
18
52.80
3
2.40
7
2.40
8
63.53
J
2
9
0.0047
26
75.92
24
69.28
7
2.42
5
2.40
17
101.00
E
2
10
0.0049
12
39.56
10
33.09
15
4.39
15
4.38
28
317.99
O
2
11
0.0051
25
74.57
23
68.46
5
2.42
3
2.40
19
101.00
L
1
12
0.0054
4
10.37
4
10.48
2
2.24
2
2.23
22
119.77
O
1
13
0.0057
13
40.42
14
38.34
5
2.42
3
2.40
20
101.00
L
2
14
0.0062
6
20.85
6
20.78
1
0.66
1
0.66
27
177.49
E
1
15
0.0063
3
8.18
2
6.12
11
3.86
11
3.85
14
79.72
J
1
16
0.0068
7
27.52
7
26.15
7
2.42
5
2.40
18
101.00
G
2
17
0.0084
14
42.86
11
34.55
18
5.59
18
5.62
25
156.12
G
1
18
0.0106
1
5.52
1
3.89
18
5.59
18
5.62
26
156.12
U
2
19
0.0124
30
91.04
28
82.40
16
4.89
17
4.83
30
540.59
U
1
20
0.0139
29
86.88
30
83.62
17
4.91
16
4.82
29
440.67
S
1
21
0.0160
28
85.05
29
83.33
22
7.82
22
7.83
23
150.35
S
2
22
0.0190
27
84.02
27
82.13
22
7.82
22
7.83
24
150.35
H
1
23
0.0349
22
70.12
25
70.38
9
3.31
9
3.31
16
86.46
H
2
24
0.0361
23
72.35
26
73.06
9
3.31
9
3.31
15
86.46
M
2
25
0.0882
16
49.50
16
45.91
28
13.79
28
13.53
6
57.53
F
2
26
0.0901
10
38.37
12
36.36
26
13.78
26
13.49
11
77.02
M
1
27
0.0904
9
28.89
8
27.15
28
13.79
28
13.53
7
57.53
F
1
28
0.0910
8
28.72
9
27.27
26
13.78
26
13.49
10
77.02
C
2
29
NA
5
10.63
5
10.70
12
4.22
12
4.20
4
46.49
C
1
30
NA
2
7.36
3
6.40
12
4.22
12
4.20
3
46.49
Table 86: Tabulation of ranked results for Class B — Left and Right Slap. Letter refers to the participant’s letter code found on the footer of this page. Sub.
# is an identifier used to differentiate between the two submissions each participant could make. The FNIR column was computed at the score threshold
that gave FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. The Identification column shows the
time used to perform a search over an enrollment set of 3 000 000, as seen in Table 19. The Search Enrollment column shows the time used to create a
search template to be used for a query, as seen in Table 77. Identification and Search Enrollment durations are reported in seconds, but were originally
recorded to microsecond precision. RAM refers to the sum of the resident set sizes of the stage one identification processes over all compute nodes after
returning from the identification stage one initialization method, as seen in Table 63. RAM is reported in gigabytes, where 1 GB is equal to 1 073 741 824
bytes. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink.
The table is sorted on the FNIR column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
Sub. #
Identification
Mean
Median
FNIR
185
Search Enrollment
Mean
Median
RAM
I
2
1
0.0009
20
60.01
16
43.86
30
19.36
30
19.36
21
108.71
Q
1
2
0.0012
14
48.85
14
42.92
20
8.89
20
8.92
1
7.54
Q
2
2
0.0012
24
71.67
24
66.01
20
8.89
20
8.92
2
7.54
I
1
2
0.0012
6
24.57
7
19.07
25
16.50
25
16.52
5
49.68
D
2
2
0.0012
19
54.37
18
46.70
24
11.90
24
11.86
13
79.43
D
1
6
0.0020
17
52.42
17
45.15
17
6.19
17
6.16
12
79.43
V
2
7
0.0024
15
49.72
19
49.30
3
3.35
7
3.36
9
63.53
E
2
7
0.0024
18
52.88
15
43.61
16
5.95
16
5.93
28
317.95
V
1
9
0.0027
10
35.51
10
34.96
3
3.35
7
3.36
8
63.53
L
1
10
0.0031
5
14.42
5
14.50
2
3.05
2
3.05
22
119.79
L
2
11
0.0033
8
28.56
9
28.59
1
0.88
1
0.88
27
177.48
J
2
11
0.0033
22
64.38
22
60.13
7
3.38
3
3.35
17
101.00
O
2
13
0.0035
21
63.87
21
60.02
5
3.38
5
3.36
20
101.00
G
2
14
0.0040
9
31.67
6
16.26
18
7.89
18
7.90
26
156.12
O
1
15
0.0041
11
37.04
11
35.64
5
3.38
5
3.36
19
101.00
E
1
16
0.0043
2
8.76
2
6.76
13
5.35
13
5.34
14
79.73
J
1
17
0.0049
7
26.31
8
25.38
7
3.38
3
3.35
18
101.00
G
1
18
0.0062
1
6.33
1
4.26
18
7.89
18
7.90
25
156.12
U
1
19
0.0099
30
88.83
28
86.60
15
5.90
15
5.82
29
440.68
S
1
20
0.0108
28
86.50
29
87.60
22
10.24
22
10.31
24
150.35
S
2
21
0.0136
29
88.70
30
88.46
22
10.24
22
10.31
23
150.35
U
2
22
0.0141
25
80.14
25
75.28
14
5.80
14
5.73
30
540.60
H
1
23
0.0203
26
82.43
26
82.66
9
4.37
9
4.37
15
86.46
H
2
24
0.0204
27
85.74
27
86.56
9
4.37
9
4.37
16
86.46
M
2
25
0.0515
23
70.45
23
65.91
26
18.36
28
18.11
6
57.53
M
1
26
0.0543
13
41.37
13
38.87
26
18.36
28
18.11
6
57.53
F
1
27
0.0591
12
39.12
12
37.56
28
18.36
26
18.07
10
77.02
F
2
27
0.0591
16
51.59
20
49.34
28
18.36
26
18.07
10
77.02
C
2
29
NA
4
10.28
4
10.21
11
5.16
11
5.13
4
46.49
C
1
30
NA
3
9.00
3
7.92
11
5.16
11
5.13
3
46.49
Table 87: Tabulation of ranked results for Class B — Identification Flats. Letter refers to the participant’s letter code found on the footer of this page. Sub.
# is an identifier used to differentiate between the two submissions each participant could make. The FNIR column was computed at the score threshold
that gave FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. The Identification column shows the
time used to perform a search over an enrollment set of 3 000 000, as seen in Table 19. The Search Enrollment column shows the time used to create a
search template to be used for a query, as seen in Table 78. Identification and Search Enrollment durations are reported in seconds, but were originally
recorded to microsecond precision. RAM refers to the sum of the resident set sizes of the stage one identification processes over all compute nodes after
returning from the identification stage one initialization method, as seen in Table 63. RAM is reported in gigabytes, where 1 GB is equal to 1 073 741 824
bytes. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink.
The table is sorted on the FNIR column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
186
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
Sub. #
Identification
Mean
Median
FNIR
Search Enrollment
Mean
Median
RAM
I
2
1
0.0010
15
56.93
10
38.28
30
17.04
30
16.99
11
110.22
Q
1
2
0.0011
10
44.75
9
36.86
20
9.14
20
9.11
2
13.33
D
2
2
0.0011
25
84.92
26
74.36
24
13.69
24
13.66
15
132.37
Q
2
4
0.0013
9
43.67
8
35.35
20
9.14
20
9.11
1
13.33
I
1
4
0.0013
23
79.70
16
52.41
29
16.71
29
16.67
4
84.42
D
1
6
0.0015
28
87.07
25
73.93
17
7.12
17
7.10
16
132.37
V
1
7
0.0024
13
47.76
14
47.02
7
3.69
7
3.66
13
121.80
V
2
7
0.0024
17
63.19
19
62.26
7
3.69
7
3.66
12
121.79
O
1
9
0.0025
21
73.31
23
68.54
4
3.63
3
3.61
19
181.00
O
2
10
0.0027
24
82.13
24
73.41
4
3.63
3
3.61
22
181.00
J
2
10
0.0027
26
85.17
28
77.75
6
3.64
6
3.62
21
181.00
J
1
12
0.0047
12
46.32
12
43.57
3
3.62
5
3.62
20
181.00
E
2
13
0.0048
27
85.50
17
52.91
12
5.21
12
5.20
28
573.25
E
1
14
0.0088
2
15.94
2
12.77
11
5.20
11
5.20
14
127.39
L
2
15
0.0095
6
21.26
6
23.53
1
3.51
1
3.48
27
274.95
L
1
16
0.0102
3
17.53
3
17.51
1
3.51
1
3.48
23
192.19
U
2
17
0.0155
20
67.98
22
68.30
15
6.13
15
5.91
29
811.45
U
1
18
0.0163
22
75.14
27
74.94
15
6.13
15
5.91
29
811.45
H
2
19
0.0275
18
64.90
20
65.36
9
5.16
9
5.16
18
144.02
G
2
20
0.0276
7
37.15
7
23.69
18
8.35
18
8.32
26
274.35
H
1
20
0.0276
19
67.91
21
68.07
9
5.16
9
5.16
17
144.02
S
1
22
0.0311
29
87.76
29
86.56
22
10.91
22
10.87
24
260.37
G
1
23
0.0368
1
11.21
1
7.71
18
8.35
18
8.32
25
274.35
C
2
24
0.0711
4
18.31
5
18.38
13
5.84
13
5.80
10
92.86
F
1
25
0.0734
14
50.99
15
51.69
27
15.96
27
15.87
7
90.93
F
2
25
0.0734
16
60.34
18
60.86
27
15.96
27
15.87
6
90.93
M
2
27
0.0826
11
45.96
13
46.70
25
15.91
25
15.83
8
90.93
M
1
28
0.0934
8
42.60
11
43.37
25
15.91
25
15.83
5
90.93
S
2
29
0.1680
30
89.24
30
91.74
22
10.91
22
10.87
3
61.04
C
1
30
NA
5
21.04
4
18.30
13
5.84
13
5.80
9
92.86
Table 88: Tabulation of ranked results for Class C — Ten-Finger Plain-to-Plain. Letter refers to the participant’s letter code found on the footer of this
page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. The FNIR column was computed at the score
threshold that gave FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed. The Identification column
shows the time used to perform a search over an enrollment set of 5 000 000, as seen in Table 20. The Search Enrollment column shows the time used
to create a search template to be used for a query, as seen in Table 79. Identification and Search Enrollment durations are reported in seconds, but were
originally recorded to microsecond precision. RAM refers to the sum of the resident set sizes of the stage one identification processes over all compute
nodes after returning from the identification stage one initialization method, as seen in Table 64. RAM is reported in gigabytes, where 1 GB is equal to
1 073 741 824 bytes. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the
worst in pink. The table is sorted on the FNIR column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Letter
Sub. #
Identification
Mean
Median
FNIR
187
Search Enrollment
Mean
Median
RAM
I
1
1
0.0013
24
79.04
15
54.56
25
20.21
25
20.23
4
113.80
Q
2
2
0.0014
27
83.35
25
74.22
17
12.95
17
13.06
2
20.22
I
2
2
0.0014
9
40.53
7
30.82
24
18.72
24
18.73
11
137.03
D
1
4
0.0015
17
65.97
17
58.92
23
17.44
21
17.39
10
132.40
Q
1
5
0.0017
26
83.25
26
74.40
17
12.95
17
13.06
1
20.22
V
1
5
0.0017
10
40.94
10
40.74
9
8.48
9
8.50
17
234.03
D
2
7
0.0018
30
86.39
27
74.97
30
30.43
30
30.47
9
132.37
V
2
8
0.0019
16
65.47
18
65.07
9
8.48
9
8.50
18
234.03
O
2
9
0.0033
19
72.55
21
69.30
6
6.80
6
6.71
23
303.13
J
2
9
0.0033
20
72.62
19
67.98
8
6.82
8
6.74
24
303.13
O
1
11
0.0034
15
59.01
14
54.42
6
6.80
6
6.71
25
303.13
E
2
12
0.0050
14
57.02
11
42.73
11
9.89
11
9.91
30
930.48
J
1
13
0.0051
8
36.09
9
33.02
5
6.78
5
6.69
22
303.13
L
2
14
0.0083
3
19.52
5
20.25
3
4.36
3
4.36
26
367.82
C
2
15
0.0085
5
25.91
6
21.97
13
10.79
13
10.69
15
183.36
C
1
16
0.0094
4
25.25
4
20.15
13
10.79
13
10.69
14
183.36
L
1
17
0.0097
7
31.42
8
31.68
3
4.36
3
4.36
21
280.49
E
1
18
0.0106
1
9.52
2
8.63
12
9.90
12
9.92
16
191.66
H
2
19
0.0199
29
84.26
29
84.50
15
12.08
15
12.17
12
144.02
H
1
20
0.0201
28
84.14
30
84.51
15
12.08
15
12.17
13
144.02
G
2
21
0.0333
6
30.61
3
19.46
19
15.55
19
15.50
19
261.29
U
2
22
0.0351
23
74.39
24
72.48
1
2.94
1
2.87
28
806.17
U
1
23
0.0358
25
82.76
28
82.61
1
2.94
1
2.87
28
806.17
G
1
24
0.0447
2
11.67
1
7.78
19
15.55
19
15.50
20
261.29
F
1
25
0.0536
13
55.63
16
55.79
28
21.07
28
20.92
7
130.29
F
2
25
0.0536
18
68.61
20
68.20
28
21.07
28
20.92
6
130.29
M
2
27
0.0716
12
48.71
13
48.80
26
21.02
26
20.89
5
130.29
M
1
28
0.0783
11
47.38
12
48.08
26
21.02
26
20.89
8
130.29
S
1
29
0.0860
22
74.01
22
69.71
21
17.43
22
17.57
27
382.88
S
2
30
0.2462
21
72.95
23
70.77
21
17.43
22
17.57
3
78.28
Table 89: Tabulation of ranked results for Class C — Ten-Finger Rolled-to-Rolled. Letter refers to the participant’s letter code found on the footer of this
page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. The FNIR column was computed at the
score threshold that gave FPIR = 10−3 . The Identification column shows the time used to perform a search over an enrollment set of 5 000 000, as seen
in Table 20. The Search Enrollment column shows the time used to create a search template to be used for a query, as seen in Table 80. Identification and
Search Enrollment durations are reported in seconds, but were originally recorded to microsecond precision. RAM refers to the sum of the resident set
sizes of the stage one identification processes over all compute nodes after returning from the identification stage one initialization method, as seen in
Table 65. RAM is reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s column-wise
ranking, with the best performance shaded in green and the worst in pink. The table is sorted on the FNIR column-wise ranking.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
188
J
FPVTE – F INGERPRINT M ATCHING
Relative Combined Results
The tables in Appendix I show the detailed values for FNIR at a fixed FPIR of 10−3 , identification time, search template
enrollment time, and RAM consumption. While the column-wise ranking for each variable is given, it can be difficult to visualize relative comparisons among submissions for each of these values. Use of star plots can help with this visualization.
For more information on how to read star plots, please refer to the explanation in Appendix K.
Class A plots are in Figures 99 through 101, Class B plots are in Figures 102 through 105, and Class C plots are in Figures 106
through 107.
Some notable observations based on the plotted shape include that:
. For almost all finger combinations, I uses the longest enrollment times while providing the highest accuracy.
. D’s submission strike a balance between identification time and enrollment time in order to provide high accuracy.
. Class A results are nearly identical, regardless of which index finger combinations are used.
. J1, L1, Q, O1, and V1 provide high accuracy while limiting time and computational resources for most finger combinations.
. Shapes formed by participants appear to remain similar regardless of finger combination.
. Tradeoffs between a participant’s two submissions are very easily seen. For instance, in class Class A, V1 and S1 use
significantly shorter identification times than V2 and S2 respectively, but result in similar relative accuracies.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
189
E2
FNIR
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
H2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
Identification Time
●
●
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
Identification Time
Identification Time
●
●
Enrollment Time
RAM Consumption
H1
RAM Consumption
Enrollment Time
G2
RAM Consumption
Enrollment Time
G1
RAM Consumption
Enrollment Time
F2
RAM Consumption
Enrollment Time
F1
RAM Consumption
Enrollment Time
●
●
●
●
●
K2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
Identification Time
●
●
Identification Time
●
Identification Time
●
●
Enrollment Time
RAM Consumption
K1
RAM Consumption
Enrollment Time
J2
RAM Consumption
Enrollment Time
J1
RAM Consumption
Enrollment Time
I2
RAM Consumption
Enrollment Time
I1
RAM Consumption
Enrollment Time
●
Identification Time
●
●
Identification Time
●
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
RAM Consumption
E1
FNIR
RAM Consumption
D2
FNIR
RAM Consumption
D1
FNIR
RAM Consumption
C2
FNIR
RAM Consumption
C1
FNIR
Identification Time
RAM Consumption
FPVTE – F INGERPRINT M ATCHING
●
●
●
●
O1
O2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
Enrollment Time
●
●
●
Enrollment Time
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
S2
FNIR
●
Enrollment Time
●
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
Identification Time
Enrollment Time
●
Identification Time
●
●
Identification Time
●
Identification Time
●
●
Identification Time
●
RAM Consumption
S1
FNIR
RAM Consumption
Q2
FNIR
RAM Consumption
Q1
FNIR
RAM Consumption
P2
FNIR
RAM Consumption
P1
FNIR
Identification Time
RAM Consumption
Enrollment Time
●
●
Identification Time
●
●
Identification Time
Enrollment Time
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
Enrollment Time
RAM Consumption
M2
RAM Consumption
Enrollment Time
M1
RAM Consumption
Enrollment Time
L2
RAM Consumption
Enrollment Time
L1
RAM Consumption
Enrollment Time
Identification Time
RAM Consumption
●
Enrollment Time
●
●
●
●
Enrollment Time
T1
T2
U1
U2
V1
V2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
Enrollment Time
●
●
Enrollment Time
●
●
●
●
Enrollment Time
RAM Consumption
●
RAM Consumption
RAM Consumption
RAM Consumption
RAM Consumption
●
●
●
●
●
●
Identification Time
Enrollment Time
●
Identification Time
●
●
●
Identification Time
Enrollment Time
●
●
Identification Time
●
●
●
Identification Time
●
Identification Time
RAM Consumption
●
Enrollment Time
Figure 99: Star plot of combined results for Class A — Left Index. The values in this plot have been independently scaled from 0 to 1 from the values
printed in Table 81, with the exception of FNIR, whose log10 values were scaled. Any values printed as NA were set to 1 before scaling. The intersection
point of a radius and the circumference of a circle indicate the scaled values 0, 0.5, and 1, with 1 resting on the outer dashed-gray circle. The title above
each plot represents the participant’s letter code found on the footer of this page and an identifier used to differentiate between the two submissions
each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
E2
FNIR
●
●
Enrollment Time
●
●
●
●
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
Enrollment Time
H2
FNIR
●
●
●
●
●
●
●
●
●
●
●
Identification Time
●
●
Identification Time
●
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
RAM Consumption
H1
FNIR
RAM Consumption
G2
FNIR
RAM Consumption
G1
FNIR
RAM Consumption
F2
FNIR
RAM Consumption
F1
FNIR
Identification Time
RAM Consumption
Enrollment Time
●
●
●
●
●
●
●
K2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
Identification Time
●
●
Identification Time
●
Identification Time
●
●
Enrollment Time
RAM Consumption
K1
RAM Consumption
Enrollment Time
J2
RAM Consumption
Enrollment Time
J1
RAM Consumption
Enrollment Time
I2
RAM Consumption
Enrollment Time
I1
RAM Consumption
Enrollment Time
●
Identification Time
●
●
Identification Time
Enrollment Time
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
RAM Consumption
E1
FNIR
RAM Consumption
D2
FNIR
RAM Consumption
D1
FNIR
RAM Consumption
C2
FNIR
RAM Consumption
C1
FNIR
Identification Time
RAM Consumption
190
●
●
●
●
O2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
S1
S2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
Enrollment Time
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
Identification Time
Enrollment Time
●
●
Identification Time
●
●
Identification Time
Enrollment Time
●
●
Identification Time
●
●
Identification Time
●
●
Enrollment Time
RAM Consumption
Enrollment Time
Q2
RAM Consumption
Enrollment Time
Q1
RAM Consumption
Enrollment Time
P2
RAM Consumption
Enrollment Time
P1
RAM Consumption
Enrollment Time
●
Identification Time
●
●
Identification Time
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
Enrollment Time
RAM Consumption
O1
RAM Consumption
M2
RAM Consumption
Enrollment Time
M1
RAM Consumption
Enrollment Time
L2
RAM Consumption
Enrollment Time
L1
Identification Time
RAM Consumption
Enrollment Time
Identification Time
RAM Consumption
●
Enrollment Time
●
●
●
●
Enrollment Time
T1
T2
U1
U2
V1
V2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
Enrollment Time
●
●
Enrollment Time
●
●
●
●
Enrollment Time
RAM Consumption
●
RAM Consumption
RAM Consumption
RAM Consumption
RAM Consumption
●
●
●
●
●
●
Identification Time
Enrollment Time
●
Identification Time
●
●
●
Identification Time
Enrollment Time
●
●
Identification Time
●
●
●
Identification Time
●
Identification Time
RAM Consumption
●
Enrollment Time
Figure 100: Star plot of combined results for Class A — Right Index. The values in this plot have been independently scaled from 0 to 1 from the values
printed in Table 82, with the exception of FNIR, whose log10 values were scaled. The intersection point of a radius and the circumference of a circle
indicate the scaled values 0, 0.5, and 1, with 1 resting on the outer dashed-gray circle. The title above each plot represents the participant’s letter code
found on the footer of this page and an identifier used to differentiate between the two submissions each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
191
E2
FNIR
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
H2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
K2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
Identification Time
Identification Time
●
●
Enrollment Time
RAM Consumption
K1
RAM Consumption
Enrollment Time
J2
RAM Consumption
Enrollment Time
J1
RAM Consumption
Enrollment Time
I2
RAM Consumption
Enrollment Time
I1
RAM Consumption
Enrollment Time
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
Identification Time
●
Enrollment Time
RAM Consumption
H1
RAM Consumption
Enrollment Time
G2
RAM Consumption
Enrollment Time
G1
RAM Consumption
Enrollment Time
F2
RAM Consumption
Enrollment Time
F1
RAM Consumption
Enrollment Time
Identification Time
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
RAM Consumption
E1
FNIR
RAM Consumption
D2
FNIR
RAM Consumption
D1
FNIR
RAM Consumption
C2
FNIR
RAM Consumption
C1
FNIR
Identification Time
RAM Consumption
FPVTE – F INGERPRINT M ATCHING
●
●
●
●
O1
O2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
Enrollment Time
Enrollment Time
●
●
●
Enrollment Time
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
S2
FNIR
●
●
●
●
●
Enrollment Time
Enrollment Time
●
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
●
●
●
●
Identification Time
Enrollment Time
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
Identification Time
●
RAM Consumption
S1
FNIR
RAM Consumption
Q2
FNIR
RAM Consumption
Q1
FNIR
RAM Consumption
P2
FNIR
RAM Consumption
P1
FNIR
Identification Time
RAM Consumption
●
Identification Time
●
●
●
Identification Time
Enrollment Time
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
Enrollment Time
RAM Consumption
M2
RAM Consumption
Enrollment Time
M1
RAM Consumption
Enrollment Time
L2
RAM Consumption
Enrollment Time
L1
RAM Consumption
Enrollment Time
Identification Time
RAM Consumption
●
Enrollment Time
Enrollment Time
T1
T2
U1
U2
V1
V2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
Enrollment Time
●
●
Enrollment Time
●
●
●
●
Enrollment Time
RAM Consumption
●
RAM Consumption
RAM Consumption
RAM Consumption
RAM Consumption
●
●
●
●
●
●
Identification Time
Enrollment Time
●
Identification Time
●
●
●
Identification Time
Enrollment Time
●
●
Identification Time
●
●
●
Identification Time
●
Identification Time
RAM Consumption
●
Enrollment Time
Figure 101: Star plot of combined results for Class A — Left and Right Index. The values in this plot have been independently scaled from 0 to 1 from
the values printed in Table 21, with the exception of FNIR, whose log10 values were scaled. Any values printed as NA were set to 1 before scaling. The
intersection point of a radius and the circumference of a circle indicate the scaled values 0, 0.5, and 1, with 1 resting on the outer dashed-gray circle.
The title above each plot represents the participant’s letter code found on the footer of this page and an identifier used to differentiate between the two
submissions each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
192
FPVTE – F INGERPRINT M ATCHING
●
●
●
●
●
●
●
●
●
●
●
●
●
●
H2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
J2
L1
L2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
RAM Consumption
●
Identification Time
●
●
Enrollment Time
Identification Time
●
Identification Time
●
●
Identification Time
●
Identification Time
Identification Time
●
●
RAM Consumption
Enrollment Time
J1
RAM Consumption
Enrollment Time
I2
RAM Consumption
Enrollment Time
I1
RAM Consumption
●
Enrollment Time
RAM Consumption
●
Enrollment Time
Identification Time
●
Identification Time
●
Identification Time
●
Identification Time
●
Identification Time
Identification Time
●
●
Enrollment Time
RAM Consumption
H1
RAM Consumption
Enrollment Time
G2
RAM Consumption
Enrollment Time
G1
RAM Consumption
Enrollment Time
F2
RAM Consumption
Enrollment Time
F1
RAM Consumption
Enrollment Time
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
RAM Consumption
E2
FNIR
RAM Consumption
E1
FNIR
RAM Consumption
D2
FNIR
RAM Consumption
D1
FNIR
RAM Consumption
C2
FNIR
RAM Consumption
C1
FNIR
●
●
●
●
●
●
Q1
Q2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Enrollment Time
Enrollment Time
Enrollment Time
S2
U1
U2
V1
V2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
Enrollment Time
●
Enrollment Time
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
RAM Consumption
●
●
●
●
●
Identification Time
●
●
Enrollment Time
Identification Time
Enrollment Time
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
●
RAM Consumption
Enrollment Time
S1
RAM Consumption
Enrollment Time
RAM Consumption
●
RAM Consumption
●
Identification Time
RAM Consumption
●
Identification Time
●
●
Identification Time
●
Identification Time
●
Identification Time
●
Identification Time
Identification Time
●
Enrollment Time
RAM Consumption
O2
RAM Consumption
Enrollment Time
O1
RAM Consumption
Enrollment Time
M2
RAM Consumption
Enrollment Time
M1
RAM Consumption
Enrollment Time
RAM Consumption
Enrollment Time
Enrollment Time
Figure 102: Star plot of combined results for Class B — Left Slap. The values in this plot have been independently scaled from 0 to 1 from the values
printed in Table 84, with the exception of FNIR, whose log10 values were scaled. The intersection point of a radius and the circumference of a circle
indicate the scaled values 0, 0.5, and 1, with 1 resting on the outer dashed-gray circle. The title above each plot represents the participant’s letter code
found on the footer of this page and an identifier used to differentiate between the two submissions each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
193
●
●
●
●
●
●
●
●
●
●
●
●
●
●
H2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
J2
L1
L2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
RAM Consumption
●
●
●
Identification Time
●
●
Enrollment Time
Identification Time
●
Identification Time
●
●
Identification Time
●
Identification Time
Identification Time
●
●
RAM Consumption
Enrollment Time
J1
RAM Consumption
Enrollment Time
I2
RAM Consumption
Enrollment Time
I1
RAM Consumption
●
Enrollment Time
RAM Consumption
●
Enrollment Time
●
Identification Time
●
Identification Time
●
Identification Time
●
Identification Time
●
Identification Time
Identification Time
●
●
Enrollment Time
RAM Consumption
H1
RAM Consumption
Enrollment Time
G2
RAM Consumption
Enrollment Time
G1
RAM Consumption
Enrollment Time
F2
RAM Consumption
Enrollment Time
F1
RAM Consumption
Enrollment Time
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
RAM Consumption
E2
FNIR
RAM Consumption
E1
FNIR
RAM Consumption
D2
FNIR
RAM Consumption
D1
FNIR
RAM Consumption
C2
FNIR
RAM Consumption
C1
FNIR
●
●
●
●
●
●
Q1
Q2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Enrollment Time
Enrollment Time
Enrollment Time
S2
U1
U2
V1
V2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
Enrollment Time
●
●
Enrollment Time
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
RAM Consumption
●
●
●
●
●
●
Identification Time
Enrollment Time
●
Enrollment Time
Identification Time
●
●
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
RAM Consumption
Enrollment Time
S1
RAM Consumption
Enrollment Time
RAM Consumption
●
RAM Consumption
●
Identification Time
RAM Consumption
●
Identification Time
●
Identification Time
●
Identification Time
●
Identification Time
●
Identification Time
Identification Time
●
●
Enrollment Time
RAM Consumption
O2
RAM Consumption
Enrollment Time
O1
RAM Consumption
Enrollment Time
M2
RAM Consumption
Enrollment Time
M1
RAM Consumption
Enrollment Time
RAM Consumption
Enrollment Time
Enrollment Time
Figure 103: Star plot of combined results for Class B — Right Slap. The values in this plot have been independently scaled from 0 to 1 from the values
printed in Table 85, with the exception of FNIR, whose log10 values were scaled. The intersection point of a radius and the circumference of a circle
indicate the scaled values 0, 0.5, and 1, with 1 resting on the outer dashed-gray circle. The title above each plot represents the participant’s letter code
found on the footer of this page and an identifier used to differentiate between the two submissions each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
194
FPVTE – F INGERPRINT M ATCHING
●
●
●
●
●
●
●
●
●
●
●
●
●
●
H2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
Enrollment Time
Enrollment Time
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
L2
FNIR
●
●
●
●
●
●
●
●
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
●
Identification Time
●
Identification Time
●
●
RAM Consumption
L1
FNIR
RAM Consumption
J2
FNIR
RAM Consumption
J1
FNIR
RAM Consumption
I2
FNIR
RAM Consumption
I1
FNIR
Identification Time
RAM Consumption
●
Enrollment Time
●
●
●
Identification Time
●
Enrollment Time
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
Identification Time
Identification Time
●
●
Enrollment Time
RAM Consumption
H1
RAM Consumption
Enrollment Time
G2
RAM Consumption
Enrollment Time
G1
RAM Consumption
Enrollment Time
F2
RAM Consumption
Enrollment Time
F1
RAM Consumption
Enrollment Time
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
●
Identification Time
●
Identification Time
Identification Time
●
RAM Consumption
E2
FNIR
RAM Consumption
E1
FNIR
RAM Consumption
D2
FNIR
RAM Consumption
D1
FNIR
●
RAM Consumption
C2
FNIR
●
RAM Consumption
C1
FNIR
●
●
●
●
●
●
Q1
Q2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
Enrollment Time
Enrollment Time
●
●
●
●
●
Enrollment Time
Enrollment Time
●
●
●
●
Enrollment Time
Enrollment Time
●
Enrollment Time
●
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
Identification Time
Enrollment Time
●
Identification Time
●
●
Identification Time
●
Identification Time
●
RAM Consumption
V2
FNIR
RAM Consumption
V1
FNIR
RAM Consumption
U2
FNIR
RAM Consumption
U1
FNIR
RAM Consumption
S2
FNIR
●
●
●
S1
●
●
●
●
FNIR
Identification Time
RAM Consumption
●
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
Identification Time
●
●
Identification Time
●
Identification Time
Identification Time
●
Enrollment Time
RAM Consumption
O2
RAM Consumption
Enrollment Time
O1
RAM Consumption
Enrollment Time
M2
RAM Consumption
Enrollment Time
M1
RAM Consumption
Enrollment Time
RAM Consumption
Enrollment Time
●
●
●
Enrollment Time
Figure 104: Star plot of combined results for Class B — Left and Right Slap. The values in this plot have been independently scaled from 0 to 1 from
the values printed in Table 86, with the exception of FNIR, whose log10 values were scaled. Any values printed as NA were set to 1 before scaling. The
intersection point of a radius and the circumference of a circle indicate the scaled values 0, 0.5, and 1, with 1 resting on the outer dashed-gray circle.
The title above each plot represents the participant’s letter code found on the footer of this page and an identifier used to differentiate between the two
submissions each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
195
●
●
●
●
●
●
●
●
●
●
●
●
●
●
H2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
Enrollment Time
Enrollment Time
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
L2
FNIR
●
●
●
●
●
●
●
●
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
●
Identification Time
●
Identification Time
●
●
RAM Consumption
L1
FNIR
RAM Consumption
J2
FNIR
RAM Consumption
J1
FNIR
RAM Consumption
I2
FNIR
RAM Consumption
I1
FNIR
Identification Time
RAM Consumption
●
Enrollment Time
●
●
●
Identification Time
●
Enrollment Time
●
●
Identification Time
●
●
Identification Time
●
Identification Time
●
Identification Time
Identification Time
●
●
Enrollment Time
RAM Consumption
H1
RAM Consumption
Enrollment Time
G2
RAM Consumption
Enrollment Time
G1
RAM Consumption
Enrollment Time
F2
RAM Consumption
Enrollment Time
F1
RAM Consumption
Enrollment Time
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
●
Identification Time
●
Identification Time
Identification Time
●
RAM Consumption
E2
FNIR
RAM Consumption
E1
FNIR
RAM Consumption
D2
FNIR
RAM Consumption
D1
FNIR
●
RAM Consumption
C2
FNIR
●
RAM Consumption
C1
FNIR
●
●
●
●
●
●
Q1
Q2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
Enrollment Time
Enrollment Time
●
●
●
●
●
●
Enrollment Time
Enrollment Time
●
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
V2
FNIR
●
Enrollment Time
●
●
●
Enrollment Time
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
●
●
●
●
Identification Time
Enrollment Time
●
Identification Time
●
●
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
RAM Consumption
V1
FNIR
RAM Consumption
U2
FNIR
RAM Consumption
U1
FNIR
RAM Consumption
S2
FNIR
RAM Consumption
S1
FNIR
Identification Time
RAM Consumption
●
●
●
Identification Time
●
Identification Time
●
Identification Time
●
●
Identification Time
●
Identification Time
Identification Time
●
Enrollment Time
RAM Consumption
O2
RAM Consumption
Enrollment Time
O1
RAM Consumption
Enrollment Time
M2
RAM Consumption
Enrollment Time
M1
RAM Consumption
Enrollment Time
RAM Consumption
Enrollment Time
Enrollment Time
Figure 105: Star plot of combined results for Class B — Identification Flats. The values in this plot have been independently scaled from 0 to 1 from the
values printed in Table 22, with the exception of FNIR, whose log10 values were scaled. Any values printed as NA were set to 1 before scaling. The
intersection point of a radius and the circumference of a circle indicate the scaled values 0, 0.5, and 1, with 1 resting on the outer dashed-gray circle.
The title above each plot represents the participant’s letter code found on the footer of this page and an identifier used to differentiate between the two
submissions each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
196
FPVTE – F INGERPRINT M ATCHING
C1
C2
D1
D2
E1
E2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
RAM Consumption
●
●
RAM Consumption
●
RAM Consumption
RAM Consumption
RAM Consumption
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
Identification Time
RAM Consumption
●
●
●
●
●
●
G1
G2
H1
H2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
Enrollment Time
●
●
●
●
●
Enrollment Time
Enrollment Time
●
●
●
Enrollment Time
●
●
●
●
Enrollment Time
L2
FNIR
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Enrollment Time
O1
O2
Q1
Q2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
Enrollment Time
Enrollment Time
●
●
●
●
Enrollment Time
Enrollment Time
●
●
●
●
RAM Consumption
RAM Consumption
●
●
●
Enrollment Time
Identification Time
●
●
●
Enrollment Time
Identification Time
●
Identification Time
●
Identification Time
●
Identification Time
Identification Time
●
●
RAM Consumption
Enrollment Time
M2
RAM Consumption
Enrollment Time
M1
RAM Consumption
●
Enrollment Time
RAM Consumption
●
Enrollment Time
●
Identification Time
●
●
Identification Time
●
Identification Time
●
●
Identification Time
●
Identification Time
●
●
RAM Consumption
L1
FNIR
RAM Consumption
J2
FNIR
RAM Consumption
J1
FNIR
RAM Consumption
I2
FNIR
RAM Consumption
I1
FNIR
Identification Time
RAM Consumption
●
Enrollment Time
●
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
●
Identification Time
●
Identification Time
Identification Time
●
Enrollment Time
RAM Consumption
F2
RAM Consumption
Enrollment Time
F1
RAM Consumption
Enrollment Time
RAM Consumption
Enrollment Time
RAM Consumption
Enrollment Time
RAM Consumption
Enrollment Time
●
●
●
●
Enrollment Time
S1
S2
U1
U2
V1
V2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
Enrollment Time
Enrollment Time
●
●
Enrollment Time
●
●
●
●
Enrollment Time
RAM Consumption
●
●
RAM Consumption
●
RAM Consumption
RAM Consumption
RAM Consumption
●
●
●
●
●
Identification Time
●
●
Identification Time
Enrollment Time
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
Identification Time
RAM Consumption
●
●
Enrollment Time
Figure 106: Star plot of combined results for Class C — Ten-Finger Plain-to-Plain. The values in this plot have been independently scaled from 0 to 1
from the values printed in Table 88, with the exception of FNIR, whose log10 values were scaled. Any values printed as NA were set to 1 before scaling.
The intersection point of a radius and the circumference of a circle indicate the scaled values 0, 0.5, and 1, with 1 resting on the outer dashed-gray circle.
The title above each plot represents the participant’s letter code found on the footer of this page and an identifier used to differentiate between the two
submissions each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
197
E2
FNIR
●
●
●
●
●
●
●
●
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
●
RAM Consumption
E1
FNIR
RAM Consumption
D2
FNIR
RAM Consumption
D1
FNIR
RAM Consumption
C2
FNIR
RAM Consumption
C1
FNIR
Identification Time
RAM Consumption
FPVTE – F INGERPRINT M ATCHING
●
●
●
●
●
F2
G1
G2
H1
H2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
RAM Consumption
●
●
●
●
●
●
●
●
●
●
●
●
●
L2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Q2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Enrollment Time
Enrollment Time
Enrollment Time
Enrollment Time
Enrollment Time
S1
S2
U1
U2
V1
V2
FNIR
FNIR
FNIR
FNIR
FNIR
FNIR
Identification Time
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
●
Identification Time
Identification Time
●
Enrollment Time
RAM Consumption
Q1
RAM Consumption
Enrollment Time
O2
RAM Consumption
Enrollment Time
O1
RAM Consumption
Enrollment Time
M2
RAM Consumption
Enrollment Time
M1
RAM Consumption
Enrollment Time
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
●
●
Identification Time
●
Identification Time
Identification Time
●
●
Enrollment Time
RAM Consumption
L1
RAM Consumption
Enrollment Time
J2
RAM Consumption
Enrollment Time
J1
RAM Consumption
Enrollment Time
I2
RAM Consumption
Enrollment Time
I1
RAM Consumption
Enrollment Time
Identification Time
●
●
●
Identification Time
●
Identification Time
●
●
Identification Time
●
Identification Time
Identification Time
●
●
Enrollment Time
RAM Consumption
Enrollment Time
F1
RAM Consumption
Enrollment Time
RAM Consumption
Enrollment Time
RAM Consumption
Enrollment Time
RAM Consumption
Enrollment Time
Enrollment Time
●
Enrollment Time
Enrollment Time
●
●
Enrollment Time
●
●
●
●
Enrollment Time
RAM Consumption
●
RAM Consumption
RAM Consumption
RAM Consumption
RAM Consumption
●
●
●
●
●
●
Identification Time
●
●
Identification Time
Enrollment Time
●
●
Identification Time
●
●
●
Identification Time
●
Identification Time
●
Identification Time
RAM Consumption
●
Enrollment Time
Figure 107: Star plot of combined results for Class C — Ten-Finger Rolled-to-Rolled. The values in this plot have been independently scaled from 0 to 1
from the values printed in Table 23, with the exception of FNIR, whose log10 values were scaled. The intersection point of a radius and the circumference
of a circle indicate the scaled values 0, 0.5, and 1, with 1 resting on the outer dashed-gray circle. The title above each plot represents the participant’s
letter code found on the footer of this page and an identifier used to differentiate between the two submissions each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
198
K
FPVTE – F INGERPRINT M ATCHING
Relative Accuracy and Number of Fingers
In Section 7, DETs, tables, and other data visualization methods were used to show exact FNIR values at FPIR = 10−3 .
While this data is important for exact comparisons, it is difficult to quickly compare relative accuracy among submissions,
or to see how a submission fares across finger combinations. To facilitate these comparisons, star plots [1] (also called
spider or radar plots) are included in this section. These plots are used to quickly examine relative values among multiple
variables.
In each star plot, values are plotted along multiple radii, where each radius represents a single variable. The plot area is
delineated with three circles. The points on the circumference of these circles indicate different values where they intersect
the radii. The intersection with the smallest circle represents 0, the next-largest circle (dashed-blue) represents 0.5, and the
largest circle (dashed-gray) represents 1. All values plotted have been scaled between 0 and 1 against other values for the
same variable in order to fit within the largest circle. These plotted values were then connected to adjacent plotted values,
creating a polygon. Many traits may be quickly inferred by the shape of these polygons:
. Regular polygons indicate submissions that have similar accuracy for all plotted variables.
. A submission that is the most accurate in terms of every variable would inscribe the center circle and the least
accurate would inscribe the largest circle.
. The smaller the polygon, the more accurate the submission.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
K.1
199
Class A
Relative FNIR results for left index, right index, and left and right index searches are shown in Figure 108. Note that the
enrollment set sizes differed between single index and two-index searches.
Some notable observations include:
. The most accurate submissions (D, I, Q, V) are instantly recognizable by their small shape.
. Most submissions have near-identical accuracy, regardless of being provided one or two index fingers, as indicated
by the equilateral triangle drawn. This may have impacts on data collection and storage. Extreme examples include
D, K, M, and V.
. No participants appear to have wildly-differing accuracy among any of the index finger combinations.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
E2
Left Index
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
S2
Left Index
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Right Index
Right Index
Right Index
Right Index
●
Right Index
Right Index
●
●
Left and Right Index
S1
Left Index
Left and Right Index
Q2
Left Index
Left and Right Index
Q1
Left Index
Left and Right Index
P2
Left Index
Left and Right Index
P1
Left Index
Right Index
●
Right Index
●
●
Right Index
●
Right Index
Right Index
Right Index
●
●
Left and Right Index
O2
Left Index
Left and Right Index
O1
Left Index
Left and Right Index
M2
Left Index
Left and Right Index
M1
Left Index
Left and Right Index
L2
Left Index
Left and Right Index
L1
Left Index
Right Index
●
Right Index
●
Right Index
Right Index
Right Index
Right Index
●
●
Left and Right Index
K2
Left Index
Left and Right Index
K1
Left Index
Left and Right Index
J2
Left Index
Left and Right Index
J1
Left Index
Left and Right Index
I2
Left Index
Left and Right Index
I1
Left Index
Right Index
●
●
Right Index
●
●
Right Index
●
●
Right Index
Right Index
●
●
Left and Right Index
H2
Left Index
Left and Right Index
H1
Left Index
Left and Right Index
G2
Left Index
Left and Right Index
G1
Left Index
Left and Right Index
F2
Left Index
Right Index
●
F1
Right Index
Left and Right Index
●
●
●
Left Index
●
Left and Right Index
●
●
●
Right Index
●
●
Right Index
●
●
Right Index
Right Index
●
●
Left and Right Index
E1
Left Index
Left and Right Index
D2
Left Index
Left and Right Index
D1
Left Index
Left and Right Index
C2
Left Index
Left and Right Index
C1
Left Index
Right Index
Left and Right Index
200
T1
T2
U1
U2
V1
V2
Left Index
Left Index
Left Index
Left Index
Left Index
Left Index
●
●
●
Left and Right Index
●
Left and Right Index
●
●
●
●
Right Index
●
Left and Right Index
●
Right Index
Left and Right Index
●
●
Right Index
Left and Right Index
●
●
Right Index
●
●
Right Index
●
Right Index
Left and Right Index
●
Figure 108: Star plots of relative accuracy for Class A. Values plotted along each radius are log10 of values from Tables 7 through 9, scaled independently
from 0 to 1. Values printed as NA in those tables were set to 1 prior to scaling. 30 000 searches were run against an enrollment set of 100 000 subjects for
left index and right index, and 1 600 000 subjects for left and right index. The intersection point of a radius and the circumference of a circle indicate the
scaled values 0, 0.5, and 1, with 1 resting on the outer dashed-gray circle. The title above each plot represents the participant’s letter code found on the
footer of this page and an identifier used to differentiate between the two submissions each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
K.2
201
Class B
Relative FNIR results for left slap, right slap, left and right slap, and IDFlat searches are shown in Figure 109. Some notable
observations include:
. The most accurate submissions (I and Q) are instantly recognizable by their small shape.
. The most accurate submissions are also square, meaning they perform equally well for all finger combinations.
. Most submissions appear to skew towards slightly worse accuracy on left slap and right slap searches, as indicated
by the obtuse angles at the vertices for IDFlats and left and right slap.
. Some submissions, like C, F, H, M, and S, vary significantly as the number of fingers at their disposal changes.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
E2
Left Slap
●
●
●
●
●
●
●
●
●
●
●
●
Left and Right Slap
G1
G2
H1
H2
Left Slap
Left Slap
Left Slap
Left Slap
Left Slap
Left Slap
●
●
●
●
Left and Right Slap
●
●
Left and Right Slap
●
●
●
Identification Flats
●
Left and Right Slap
●
●
●
Left and Right Slap
L2
Left Slap
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Q1
Q2
Left Slap
Left Slap
Left Slap
Left Slap
Left Slap
Left Slap
●
●
Left and Right Slap
●
●
●
Left and Right Slap
●
●
●
●
Left and Right Slap
Right Slap
Left and Right Slap
●
●
Right Slap
●
●
●
Right Slap
Left and Right Slap
●
Right Slap
●
●
Right Slap
●
Right Slap
●
●
Left and Right Slap
Identification Flats
Left and Right Slap
O2
Identification Flats
Left and Right Slap
O1
Identification Flats
Left and Right Slap
M2
Identification Flats
Left and Right Slap
M1
Identification Flats
Left and Right Slap
●
●
●
●
Left and Right Slap
V2
Left Slap
●
●
●
Left and Right Slap
●
●
●
Left and Right Slap
●
●
●
●
Left and Right Slap
●
●
●
●
Right Slap
Left and Right Slap
●
Right Slap
●
●
Right Slap
Left and Right Slap
●
Right Slap
●
●
Right Slap
●
Right Slap
●
●
Identification Flats
V1
Left Slap
Identification Flats
U2
Left Slap
Identification Flats
U1
Left Slap
Identification Flats
S2
Left Slap
Identification Flats
S1
Left Slap
●
Right Slap
●
Right Slap
●
●
●
Right Slap
●
Right Slap
●
●
Identification Flats
L1
Left Slap
Identification Flats
J2
Left Slap
Identification Flats
J1
Left Slap
Identification Flats
I2
Left Slap
Identification Flats
I1
Left Slap
Right Slap
Identification Flats
Left and Right Slap
●
●
Right Slap
●
●
Left and Right Slap
Right Slap
●
●
Right Slap
●
●
Right Slap
●
Right Slap
●
●
Identification Flats
Left and Right Slap
F2
Identification Flats
Left and Right Slap
F1
Identification Flats
●
Left and Right Slap
Identification Flats
●
Right Slap
Identification Flats
●
●
●
Left and Right Slap
Left and Right Slap
Identification Flats
●
Right Slap
●
●
Right Slap
●
Right Slap
●
Right Slap
●
Right Slap
●
Identification Flats
E1
Left Slap
Identification Flats
D2
Left Slap
Identification Flats
D1
Left Slap
Identification Flats
C2
Left Slap
Identification Flats
C1
Left Slap
Right Slap
Identification Flats
FPVTE – F INGERPRINT M ATCHING
Right Slap
Identification Flats
202
Left and Right Slap
Figure 109: Star plots of relative accuracy for Class B. Values plotted along each radius are log10 of values from Tables 10 through 13, scaled independently
from 0 to 1. Values printed as NA in those tables were set to 1 prior to scaling. 30 000 searches were run against an enrollment set of 3 000 000 subjects.
The intersection point of a radius and the circumference of a circle indicate the scaled values 0, 0.5, and 1, with 1 resting on the outer dashed-gray circle.
The title above each plot represents the participant’s letter code found on the footer of this page and an identifier used to differentiate between the two
submissions each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
K.3
203
Class C
Relative FNIR results for ten-finger plain-to-plain, ten-finger rolled-to-rolled, and ten-finger plain-to-rolled searches are
shown in Figure 110. Some notable observations include:
. The most accurate submissions (D, I, and Q) are instantly recognizable by their small shape and are seemingly
equilateral, indicating that they perform equally well for all finger combinations.
. Many submissions had lower accuracy with plain-to-rolled searching over searching homogeneous impressions,
including F, G, and M.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
204
FPVTE – F INGERPRINT M ATCHING
C1
C2
D1
D2
E1
E2
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
●
●
●
Plain−to−Rolled
Plain−to−Rolled
Plain−to−Rolled
Plain−to−Rolled
F2
G1
G2
H1
H2
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
●
●
●
●
●
Plain−to−Rolled
Plain−to−Rolled
Plain−to−Rolled
●
●
●
●
I1
I2
J1
J2
L1
L2
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
●
●
Rolled−to−Rolled
●
●
Rolled−to−Rolled
●
●
Rolled−to−Rolled
●
●
Rolled−to−Rolled
Rolled−to−Rolled
Rolled−to−Rolled
●
●
Plain−to−Rolled
F1
Plain−to−Rolled
Plain−to−Rolled
●
●
Plain−to−Plain
●
Plain−to−Rolled
●
●
Rolled−to−Rolled
●
●
Rolled−to−Rolled
Rolled−to−Rolled
●
●
Rolled−to−Rolled
Rolled−to−Rolled
Rolled−to−Rolled
●
●
●
●
●
Plain−to−Rolled
●
●
●
Plain−to−Rolled
Plain−to−Rolled
Plain−to−Rolled
Plain−to−Rolled
M1
M2
O1
O2
Q1
Q2
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
●
●
●
●
Plain−to−Rolled
Plain−to−Rolled
Plain−to−Rolled
Plain−to−Rolled
●
●
●
Rolled−to−Rolled
●
●
Rolled−to−Rolled
●
Rolled−to−Rolled
●
Rolled−to−Rolled
Rolled−to−Rolled
●
Rolled−to−Rolled
●
●
●
Plain−to−Rolled
Plain−to−Rolled
●
●
Plain−to−Plain
●
Plain−to−Rolled
●
●
Rolled−to−Rolled
●
●
Rolled−to−Rolled
●
●
Rolled−to−Rolled
●
Rolled−to−Rolled
●
Rolled−to−Rolled
Rolled−to−Rolled
●
●
●
Plain−to−Rolled
●
●
●
●
S1
S2
U1
U2
V1
V2
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
Plain−to−Plain
●
●
●
●
●
Plain−to−Rolled
●
Plain−to−Rolled
Plain−to−Rolled
Plain−to−Rolled
Plain−to−Rolled
●
●
●
Rolled−to−Rolled
●
●
●
Rolled−to−Rolled
●
Rolled−to−Rolled
●
Rolled−to−Rolled
●
Rolled−to−Rolled
●
Rolled−to−Rolled
Plain−to−Rolled
●
●
Figure 110: Star plots of relative accuracy for Class C. Values plotted along each radius are log10 of values from Tables 14 through 16, scaled independently from 0 to 1. Values printed as NA in those tables were set to 1 prior to scaling. 30 000 searches were run against an enrollment set of 5 000 000
subjects. The intersection point of a radius and the circumference of a circle indicate the scaled values 0, 0.5, and 1, with 1 resting on the outer dashedgray circle. The title above each plot represents the participant’s letter code found on the footer of this page and an identifier used to differentiate
between the two submissions each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
K.4
205
All Classes
Relative FNIR results for left and right index, IDFlat, and ten-finger rolled-to-rolled searches are shown in Figure 111.
Note that the enrollment set sizes differed between all three finger searching combinations. Some notable observations
include:
. The top-performing submissions from Class A, B, and C (D, I, Q, and V) are instantly recognizable by their small
shape.
. Many of the most accurate submissions from all three classes appear equally as accurate in each individual class.
. If a submission had trouble with a particular class in FpVTE, it appears to be Class C, as seen by the long line
segments extending from the ten-finger rolled-to-rolled vertex in most plots that do not contain a mostly equilateral
triangle.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
206
FPVTE – F INGERPRINT M ATCHING
●
●
●
●
●
●
Rolled−to−Rolled
●
●
Rolled−to−Rolled
●
●
Rolled−to−Rolled
●
●
Rolled−to−Rolled
●
●
Rolled−to−Rolled
●
Rolled−to−Rolled
●
●
●
G1
G2
H1
H2
Left and Right Index
Left and Right Index
Left and Right Index
Left and Right Index
●
●
●
●
●
Rolled−to−Rolled
Rolled−to−Rolled
Rolled−to−Rolled
●
●
●
●
I1
I2
J1
J2
L1
L2
Left and Right Index
Left and Right Index
Left and Right Index
Left and Right Index
Left and Right Index
Left and Right Index
●
●
●
●
●
●
Rolled−to−Rolled
Rolled−to−Rolled
Rolled−to−Rolled
Rolled−to−Rolled
Rolled−to−Rolled
Rolled−to−Rolled
●
●
●
Q1
Q2
Left and Right Index
Left and Right Index
●
●
●
●
●
●
Rolled−to−Rolled
●
Identification Flats
●
Identification Flats
●
Identification Flats
●
●
Identification Flats
●
Identification Flats
●
Identification Flats
●
Rolled−to−Rolled
O2
Left and Right Index
Rolled−to−Rolled
O1
Left and Right Index
●
Rolled−to−Rolled
M2
Left and Right Index
●
Rolled−to−Rolled
M1
Left and Right Index
●
●
S1
S2
U1
U2
V1
V2
Left and Right Index
Left and Right Index
Left and Right Index
Left and Right Index
Left and Right Index
Left and Right Index
●
●
●
Identification Flats
●
Identification Flats
●
●
Identification Flats
●
●
Identification Flats
●
●
Identification Flats
Identification Flats
●
Identification Flats
●
●
Identification Flats
●
●
Identification Flats
Identification Flats
●
●
Rolled−to−Rolled
●
Identification Flats
Identification Flats
●
●
Rolled−to−Rolled
F2
Left and Right Index
Rolled−to−Rolled
F1
Left and Right Index
●
Rolled−to−Rolled
Identification Flats
E2
Left and Right Index
Identification Flats
E1
Left and Right Index
Identification Flats
D2
Left and Right Index
Identification Flats
D1
Left and Right Index
Identification Flats
C2
Left and Right Index
Identification Flats
C1
Left and Right Index
Rolled−to−Rolled
Rolled−to−Rolled
Rolled−to−Rolled
Rolled−to−Rolled
Rolled−to−Rolled
●
●
●
●
Identification Flats
●
●
●
Identification Flats
●
●
Identification Flats
●
●
Identification Flats
●
●
Identification Flats
●
Identification Flats
Rolled−to−Rolled
●
Figure 111: Star plots of relative accuracy for two-index finger, IDFlat, and ten-finger rolled-to-rolled comparisons. Values plotted along each radius
are log10 of values from Tables 9, 13, and 15, scaled independently from 0 to 1. Values printed as NA in those tables were set to 1 prior to scaling.
30 000 searches were run against an enrollment set of 1 600 000 subjects for left and right index, 3 000 000 subjects for IDFlats, and 5 000 000 subjects for
ten-finger rolled-to-rolled. The intersection point of a radius and the circumference of a circle indicate the scaled values 0, 0.5, and 1, with 1 resting on
the outer dashed-gray circle. The title above each plot represents the participant’s letter code found on the footer of this page and an identifier used to
differentiate between the two submissions each participant could make.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
L
207
Combined Sorted Rankings for a Theoretical Use Case
In prior sections, numerous tables and plots were presented showcasing individual statistics of participant submissions.
Tables 91 through 98 attempt to coalesce all the presented identification statistics and categorize the overall performance
of the submissions. Note that these rankings are theoretical and do not imply quality or endorsement of any kind. Please
read the disclaimer for more information.
For each statistic, the values and overall ranks from Sections 7 through 9 are displayed. The Score column is computed
by weighting percentiles of the results per class. The weights and percentiles were chosen based on values that provided
reasonable and consistent results across all enrollment sets for all classes.
. Search Template Enrollment Time
+2B ⇐⇒ Vf ≤ P10 or
+b ⇐⇒
Vf ≤ P15
−b ⇐⇒
Vf ≥ P85
+b ⇐⇒
Vf ≤ P15
−2B ⇐⇒
Vf ≥ P85
or
. FNIR
+(N − RFNIR )
+5B ⇐⇒
. RAM Consumption
+2b ⇐⇒ Vf ≤ P10
or
or
. Identification Time
+(N − RTime )
V f ≤ P5
or
+2B ⇐⇒
Vf ≤ P10
or
+3B ⇐⇒
Vf ≤ P15
or
+B ⇐⇒
Vf ≤ P15
or
+B ⇐⇒
V f ≤ P5
or
−2B ⇐⇒
Vf ≥ P90
or
−5B ⇐⇒
Vf ≥ P95
or
−B ⇐⇒
Vf ≥ P85
−3B ⇐⇒
Vf ≥ P90
or
−B ⇐⇒
Vf ≥ P75
where N = number of participants for finger
B = max (N/2.0 , 15)
b = max (N/6.0 , 3)
Rf = submission’s rank for factor f
Vf = submission’s value for factor f
Px = xth percentile of Vf
As shown, the primary factors used in scoring were accuracy and identification speed. While enrollment speed is important, the FpVTE API limited the time that could be used, and most participants completed enrollment in a relatively
similar timeframe. Using a very high amount of RAM was penalized heavily, as greater amounts of RAM require additional compute nodes, while less RAM could allow additional identification processes to be run, decreasing search time.
High accuracy was rewarded the most of any factor—regardless of how fast or resource-friendly a submission is, it doesn’t
matter if the results are incorrect.
The color bands are indicative of the 80th percentile, 55th percentile, and below. The best performance in each category is
shaded in green and the worst in pink. Time values in the Identification and Enroll columns are reported in seconds, but
were originally recorded to microsecond precision. RAM is the sum of the resident set sizes of the stage one identification
processes over all compute nodes after returning from the identification stage one initialization method.
Some notable observations from Tables 90 through 98 include:
. Some of the most accurate submissions used such a small amount of RAM that only a single compute node was
necessary.
. The most accurate submissions did not use the entire 90 seconds to perform identifications. This dramatically reduced the score given to submissions that used the maximum amount of time but were not with the top accuracy
range.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
208
FPVTE – F INGERPRINT M ATCHING
. In Class C, there are a number of submissions that achieve an “acceptable” level of accuracy in much less time than
the most accurate submissions.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
≥ 20 seconds
< 20 seconds
Participant
Score
FNIR
Letter
Sub. #
D
1
109
1
0.0197
Identification
20
7.32
209
Enrollment
RAM
29
1.43
19
0.61
L
1
95
8
0.0625
2
0.29
1
0.05
26
1.14
V
1
82
4
0.0253
19
6.01
10
0.32
17
0.39
Q
2
79
3
0.0226
24
10.43
19
0.54
14
0.34
Q
1
74
2
0.0222
28
15.70
19
0.54
13
0.34
I
1
73
5
0.0257
22
9.94
30
1.46
20
0.70
J
1
73
14
0.0786
4
0.54
11
0.32
9
0.32
O
1
60
15
0.0818
5
0.62
4
0.27
15
0.35
E
1
59
12
0.0745
3
0.35
13
0.35
25
1.12
L
2
57
6
0.0351
12
3.32
3
0.15
28
2.32
C
1
50
24
0.1335
1
0.26
17
0.39
7
0.18
O
2
43
13
0.0766
9
1.56
4
0.27
15
0.35
T
2
43
9
0.0685
18
5.97
8
0.31
1
0.01
J
2
43
10
0.0712
7
1.25
11
0.32
9
0.32
F
2
27
18
0.1082
14
3.56
25
1.07
5
0.18
P
2
25
22
0.1272
13
3.33
15
0.36
22
0.78
F
1
24
20
0.1111
11
2.49
25
1.07
6
0.18
G
1
20
19
0.1089
21
9.74
21
0.55
18
0.57
C
2
14
25
0.1337
6
0.76
17
0.39
8
0.18
P
1
14
23
0.1308
8
1.32
15
0.36
23
0.78
S
1
10
7
0.0571
29
16.86
22
0.88
21
0.72
H
1
5
26
0.1576
15
3.61
6
0.30
11
0.33
H
2
1
27
0.1607
17
4.13
6
0.30
11
0.33
U
1
−2
21
0.1218
27
14.42
2
0.15
27
1.70
K
2
−10
16
0.0875
23
10.32
23
1.05
30
3.87
11
0.0723
30
1.11
E
2
−11
16.90
13
0.35
24
K
1
−12
17
0.0883
25
10.44
23
1.05
29
3.87
M
1
−49
29
0.2995
10
1.84
27
1.07
3
0.14
T
1
−51
30
NA
16
3.99
8
0.31
2
0.01
M
2
−55
28
0.2921
26
10.70
27
1.07
4
0.14
D
2
84
1
0.0197
3
41.84
5
1.42
3
0.61
S
2
4
4
0.0650
4
44.66
4
0.88
4
0.72
I
2
3
3
0.0278
2
40.99
6
2.15
5
1.04
V
2
−5
2
0.0252
6
65.35
2
0.32
1
0.39
G
2
−13
5
0.1086
5
54.03
3
0.55
2
0.57
U
2
−40
6
0.1178
1
24.16
1
0.15
6
1.42
Table 90: Tabulation of operationally-ranked results for Class A — Left Index. Submissions were split into two groups. The first group includes
submissions that performed searches on average in less than 20 seconds, and the second includes those that took, on average, 20 seconds or longer.
Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each
participant could make. Score refers to the sum of the scoring equations shown in Appendix L. The color bands are indicative of the 80th percentile,
55th percentile, and below of the Score column. The FNIR column was computed at the score threshold that gave FPIR = 10−3 . NA indicates that the
operations required to produce the value could not be performed. The Identification column shows the time used to perform a search over an enrollment
set of 100 000, as seen in Table 17. The Enrollment column shows the time used to create a search template to be used for a query, as seen in Table 72.
Identification and Enrollment durations are reported in seconds, but were originally recorded to microsecond precision. RAM refers to the sum of the
resident set sizes of the stage one identification processes over all compute nodes after returning from the identification stage one initialization method,
as seen in Table 60. RAM is reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s
column-wise ranking, with the best performance shaded in green and the worst in pink. The table is sorted on descending Score.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
210
FPVTE – F INGERPRINT M ATCHING
≥ 20 seconds
< 20 seconds
Participant
Score
FNIR
Identification
Enrollment
Letter
Sub. #
Q
2
111
2
0.0214
23
9.98
19
D
1
109
1
0.0190
20
7.46
L
1
95
8
0.0505
2
0.26
V
1
81
5
0.0223
18
E
1
76
11
0.0630
3
RAM
0.54
11
0.33
29
1.43
19
0.61
1
0.05
26
1.14
5.60
10
0.32
17
0.39
0.32
13
0.36
24
1.07
I
1
74
3
0.0215
22
9.83
30
1.46
20
0.69
O
1
58
17
0.0776
5
0.56
4
0.27
15
0.34
L
2
57
6
0.0295
13
3.26
3
0.16
28
2.32
J
1
55
16
0.0708
4
0.51
11
0.33
10
0.31
C
1
50
24
0.1132
1
0.24
17
0.39
7
0.17
Q
1
43
4
0.0218
28
14.24
19
0.54
11
0.33
O
2
43
13
0.0675
9
1.36
4
0.27
16
0.34
T
2
43
9
0.0562
19
5.87
8
0.31
1
0.01
J
2
41
12
0.0643
7
1.13
11
0.33
9
0.31
F
2
27
18
0.0903
15
3.60
25
1.09
5
0.17
P
2
25
22
0.1100
12
3.07
15
0.37
23
0.76
F
1
24
20
0.0933
11
2.38
25
1.09
6
0.17
G
1
20
19
0.0910
24
10.10
21
0.55
18
0.57
C
2
16
23
0.1124
6
0.69
17
0.39
8
0.18
P
1
12
25
0.1133
8
1.24
15
0.37
22
0.76
S
1
9
7
0.0442
30
16.55
22
0.88
21
0.71
H
1
5
26
0.1230
14
3.56
6
0.30
14
0.33
H
2
1
27
0.1249
17
4.05
6
0.30
13
0.33
U
1
−3
21
0.0996
27
10.76
2
0.15
27
1.56
K
2
−9
15
0.0685
25
10.27
23
1.05
29
3.87
K
1
−9
14
0.0682
26
10.42
23
1.05
30
3.87
E
2
−10
10
0.0624
29
16.27
13
0.36
25
1.07
T
1
−19
28
0.1929
16
3.73
8
0.31
1
0.01
M
1
−50
30
0.2615
10
1.78
27
1.09
3
0.14
M
2
−80
29
0.2526
21
9.39
27
1.09
3
0.14
D
2
84
1
0.0190
2
28.64
5
1.43
3
0.61
I
2
19
2
0.0214
3
40.35
6
2.15
5
1.01
S
2
4
4
0.0503
4
46.05
4
0.88
4
0.71
G
2
−13
5
0.0909
5
59.55
3
0.55
2
0.57
V
2
−21
3
0.0222
6
61.24
2
0.32
1
0.39
U
2
−40
6
0.1007
1
20.10
1
0.15
6
1.30
Table 91: Tabulation of operationally-ranked results for Class A — Right Index. Submissions were split into two groups. The first group includes
submissions that performed searches on average in less than 20 seconds, and the second includes those that took, on average, 20 seconds or longer.
Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each
participant could make. Score refers to the sum of the scoring equations shown in Appendix L. The color bands are indicative of the 80th percentile,
55th percentile, and below of the Score column. The FNIR column was computed at the score threshold that gave FPIR = 10−3 . The Identification column
shows the time used to perform a search over an enrollment set of 100 000, as seen in Table 17. The Enrollment column shows the time used to create a
search template to be used for a query, as seen in Table 73. Identification and Enrollment durations are reported in seconds, but were originally recorded
to microsecond precision. RAM refers to the sum of the resident set sizes of the stage one identification processes over all compute nodes after returning
from the identification stage one initialization method, as seen in Table 61. RAM is reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The
number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. The table is
sorted on descending Score.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
≥ 20 seconds
< 20 seconds
Participant
Score
FNIR
211
Identification
Enrollment
0.0034
5
9.29
3
0.64
5
11.77
4
0.0146
2
2.19
1
0.11
8
18.76
3
0.0143
6
8.30
Letter
Sub. #
V
1
87
1
L
1
43
RAM
J
1
24
13.64
4
0.65
3
I
1
19
2
0.0058
8
17.87
9
2.93
6
15.83
C
1
12
7
0.0368
1
2.08
5
0.78
2
4.87
O
1
9
5
0.0229
7
14.46
2
0.53
4
9.05
C
2
−3
8
0.0374
4
6.35
5
0.78
1
4.87
K
1
−44
6
0.0360
9
18.01
8
2.09
9
61.81
G
1
−69
9
0.0515
3
5.22
7
1.09
7
16.38
Q
2
109
1
0.0027
19
161.02
15
1.08
9
9.14
Q
1
107
1
0.0027
21
212.69
15
1.08
10
9.14
L
2
89
7
0.0072
3
23.42
3
0.31
26
53.98
V
2
78
3
0.0028
18
127.65
9
0.64
13
11.77
D
1
74
4
0.0030
15
70.99
25
2.84
17
18.58
E
1
71
11
0.0207
1
16.35
11
0.70
22
33.40
S
1
70
13
0.0281
2
23.00
18
1.75
14
14.82
D
2
68
4
0.0030
23
237.43
26
2.84
17
18.58
J
2
39
8
0.0143
7
33.35
10
0.65
7
8.30
O
2
36
12
0.0214
10
43.53
4
0.53
8
9.05
I
2
36
4
0.0030
25
338.88
27
4.30
21
30.83
T
2
35
19
0.0366
9
37.23
7
0.62
1
0.01
U
1
26
17
0.0336
11
45.51
1
0.30
25
44.63
P
2
21
16
0.0333
17
101.16
13
0.72
19
21.41
P
1
20
20
0.0370
14
63.65
13
0.72
20
21.42
G
2
17
15
0.0311
22
221.16
17
1.09
16
16.38
H
1
7
24
0.0686
8
36.71
5
0.60
12
9.36
F
1
5
21
0.0386
13
60.66
21
2.15
5
4.86
K
2
4
14
0.0286
6
32.93
20
2.09
27
61.81
H
2
4
23
0.0684
12
52.25
5
0.60
11
9.36
F
2
2
22
0.0412
16
73.95
21
2.15
5
4.86
U
2
−3
18
0.0358
24
240.40
1
0.30
24
37.35
S
2
−11
9
0.0195
26
495.50
18
1.75
14
14.82
E
2
−13
10
0.0202
27
518.11
11
0.70
23
33.41
T
1
−28
27
NA
4
25.68
7
0.62
1
0.01
M
2
−36
25
NA
20
171.24
23
2.16
4
3.76
M
1
−48
26
NA
5
31.44
23
2.16
3
3.75
Table 92: Tabulation of operationally-ranked results for Class A — Left and Right Index. Submissions were split into two groups. The first group
includes submissions that performed searches on average in less than 20 seconds, and the second includes those that took, on average, 20 seconds
or longer. Letter refers to the participant’s letter code found on the footer of this page. Sub. # is an identifier used to differentiate between the two
submissions each participant could make. Score refers to the sum of the scoring equations shown in Appendix L. The color bands are indicative of the
80th percentile, 55th percentile, and below of the Score column. The FNIR column was computed at the score threshold that gave FPIR = 10−3 . NA
indicates that the operations required to produce the value could not be performed. The Identification column shows the time used to perform a search
over an enrollment set of 1 600 000, as seen in Table 18. The Enrollment column shows the time used to create a search template to be used for a query,
as seen in Table 74. Identification and Enrollment durations are reported in seconds, but were originally recorded to microsecond precision. RAM refers
to the sum of the resident set sizes of the stage one identification processes over all compute nodes after returning from the identification stage one
initialization method, as seen in Table 62. RAM is reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of a value
provides the value’s column-wise ranking, with the best performance shaded in green and the worst in pink. The table is sorted on descending Score.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
212
FPVTE – F INGERPRINT M ATCHING
Participant
Score
FNIR
Identification
Enrollment
RAM
Letter
Sub. #
Q
1
126
2
0.0098
19
53.24
20
3.41
1
7.54
I
2
111
1
0.0094
13
38.26
30
7.67
21
108.71
L
1
101
16
0.0288
3
5.56
2
1.11
22
119.79
Q
2
96
3
0.0099
20
54.02
20
3.41
2
7.54
I
1
83
4
0.0116
21
55.96
25
6.50
5
49.68
V
2
83
8
0.0190
15
47.97
7
1.22
9
63.53
D
2
79
5
0.0142
22
58.10
24
4.17
12
79.43
C
1
76
22
0.0654
2
3.74
12
2.12
3
46.49
E
1
76
13
0.0259
1
2.82
11
1.95
14
79.72
V
1
71
9
0.0192
11
36.07
7
1.22
8
63.53
L
2
56
14
0.0276
5
12.37
1
0.33
27
177.49
C
2
55
21
0.0647
4
6.74
12
2.12
4
46.49
D
1
49
6
0.0163
18
52.23
14
2.15
13
79.43
O
1
42
12
0.0257
12
37.01
5
1.21
17
101.00
J
1
38
15
0.0287
7
25.61
3
1.21
20
101.00
O
2
30
11
0.0254
26
73.15
5
1.21
18
101.00
E
2
26
7
0.0187
10
34.37
15
2.21
28
318.03
G
2
21
17
0.0325
23
59.49
18
2.83
25
156.12
J
2
8
10
0.0236
27
74.38
3
1.21
19
101.00
H
1
7
23
0.0998
16
49.75
9
1.67
16
86.46
G
1
6
18
0.0371
6
16.24
18
2.83
26
156.12
H
2
2
24
0.1008
17
51.77
9
1.67
15
86.46
S
1
−5
25
0.1089
24
71.69
22
3.96
24
150.35
S
2
−22
26
0.1133
25
71.77
22
3.96
23
150.35
F
2
−31
28
0.1681
14
43.59
26
6.73
10
77.02
M
2
−47
27
0.1634
29
87.21
28
6.73
7
57.53
U
1
−48
20
0.0500
28
80.03
17
2.42
29
440.67
U
2
−48
19
0.0461
30
89.07
16
2.41
30
540.60
F
1
−57
29
0.1684
8
30.09
26
6.73
11
77.02
M
1
−59
30
0.1736
9
31.17
28
6.73
6
57.53
Table 93: Tabulation of operationally-ranked results for Class B — Left Slap. Letter refers to the participant’s letter code found on the footer of this page.
Sub. # is an identifier used to differentiate between the two submissions each participant could make. Score refers to the sum of the scoring equations
shown in Appendix L. The color bands are indicative of the 80th percentile, 55th percentile, and below of the Score column. The FNIR column was
computed at the score threshold that gave FPIR = 10−3 . The Identification column shows the time used to perform a search over an enrollment set
of 3 000 000, as seen in Table 19. The Enrollment column shows the time used to create a search template to be used for a query, as seen in Table 75.
Identification and Enrollment durations are reported in seconds, but were originally recorded to microsecond precision. RAM refers to the sum of the
resident set sizes of the stage one identification processes over all compute nodes after returning from the identification stage one initialization method,
as seen in Table 63. RAM is reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s
column-wise ranking, with the best performance shaded in green and the worst in pink. The table is sorted on descending Score.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Score
FNIR
Identification
213
Enrollment
RAM
Letter
Sub. #
I
2
113
1
0.0045
14
41.13
30
7.65
21
108.71
D
2
113
2
0.0052
21
57.19
24
4.13
12
79.43
15
0.0167
3
119.78
L
1
102
5.47
2
1.11
22
Q
2
91
3
0.0057
22
61.94
20
3.41
2
7.54
Q
1
90
3
0.0057
23
64.04
20
3.41
1
7.54
V
1
87
8
0.0106
11
35.43
3
1.19
8
63.53
I
1
82
5
0.0058
20
55.87
25
6.47
5
49.68
E
1
75
13
0.0151
1
2.97
11
1.91
14
79.72
V
2
66
9
0.0110
16
48.51
3
1.19
9
63.53
C
1
55
24
0.0403
2
3.67
12
2.09
4
46.49
D
1
50
6
0.0072
19
53.32
14
2.14
13
79.43
J
1
44
14
0.0156
7
24.14
5
1.20
18
101.00
C
2
43
23
0.0392
4
6.50
12
2.09
3
46.49
O
1
37
12
0.0142
12
35.61
7
1.20
17
101.00
L
2
37
17
0.0202
6
12.66
1
0.33
27
177.49
G
2
31
16
0.0198
13
37.28
18
2.78
25
156.12
E
2
26
7
0.0083
10
34.47
15
2.18
28
318.00
G
1
22
18
0.0212
5
10.13
18
2.78
26
156.12
S
2
14
22
0.0381
25
70.84
22
3.88
24
150.35
S
1
14
21
0.0369
24
70.54
22
3.88
23
150.35
J
2
13
10
0.0126
27
74.09
5
1.20
19
101.00
O
2
8
11
0.0132
26
73.93
7
1.20
20
101.00
H
1
3
25
0.0641
17
50.16
9
1.63
16
86.46
H
2
1
26
0.0647
18
52.84
9
1.63
15
86.46
F
2
−32
28
0.1220
15
45.92
26
6.68
10
77.02
M
2
−47
27
0.1155
29
87.41
28
6.69
6
57.53
U
1
−48
19
0.0266
30
89.26
16
2.39
29
440.67
U
2
−48
20
0.0273
28
80.11
17
2.39
30
540.59
F
1
−57
29
0.1222
8
31.87
26
6.68
11
77.02
M
1
−59
30
0.1259
9
32.03
28
6.69
7
57.53
Table 94: Tabulation of operationally-ranked results for Class B — Right Slap. Letter refers to the participant’s letter code found on the footer of this page.
Sub. # is an identifier used to differentiate between the two submissions each participant could make. Score refers to the sum of the scoring equations
shown in Appendix L. The color bands are indicative of the 80th percentile, 55th percentile, and below of the Score column. The FNIR column was
computed at the score threshold that gave FPIR = 10−3 . The Identification column shows the time used to perform a search over an enrollment set
of 3 000 000, as seen in Table 19. The Enrollment column shows the time used to create a search template to be used for a query, as seen in Table 76.
Identification and Enrollment durations are reported in seconds, but were originally recorded to microsecond precision. RAM refers to the sum of the
resident set sizes of the stage one identification processes over all compute nodes after returning from the identification stage one initialization method,
as seen in Table 63. RAM is reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s
column-wise ranking, with the best performance shaded in green and the worst in pink. The table is sorted on descending Score.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
214
FPVTE – F INGERPRINT M ATCHING
Participant
Score
FNIR
Identification
Enrollment
RAM
Letter
Sub. #
Q
1
123
2
0.0021
22
65.02
20
6.84
2
7.54
I
2
110
1
0.0015
17
46.13
30
15.33
21
108.71
Q
2
94
3
0.0022
19
53.40
20
6.84
1
7.54
I
1
92
3
0.0022
13
37.19
25
12.96
5
49.68
L
1
89
12
0.0054
4
10.48
2
2.23
22
119.77
V
1
87
7
0.0036
15
38.93
7
2.40
9
63.53
V
2
81
7
0.0036
18
52.80
7
2.40
8
63.53
D
2
76
5
0.0024
21
63.05
24
8.28
13
79.43
E
1
72
15
0.0063
2
6.12
11
3.85
14
79.72
D
1
48
6
0.0031
20
61.24
14
4.28
12
79.43
G
1
41
18
0.0106
1
3.89
18
5.62
26
156.12
L
2
40
14
0.0062
6
20.78
1
0.66
27
177.49
O
1
39
13
0.0057
14
38.34
3
2.40
20
101.00
J
1
37
16
0.0068
7
26.15
5
2.40
18
101.00
O
2
29
11
0.0051
23
68.46
3
2.40
19
101.00
G
2
29
17
0.0084
11
34.55
18
5.62
25
156.12
J
2
10
9
0.0047
24
69.28
5
2.40
17
101.00
E
2
8
10
0.0049
10
33.09
15
4.38
28
317.99
M
1
4
27
0.0904
8
27.15
28
13.53
7
57.53
F
2
4
26
0.0901
12
36.36
26
13.49
11
77.02
H
1
0
23
0.0349
25
70.38
9
3.31
16
86.46
M
2
−1
25
0.0882
16
45.91
28
13.53
6
57.53
H
2
−2
24
0.0361
26
73.06
9
3.31
15
86.46
22
0.0190
27
150.35
S
2
−4
82.13
22
7.83
24
C
1
−7
30
NA
3
6.40
12
4.20
3
46.49
S
1
−19
21
0.0160
29
83.33
22
7.83
23
150.35
F
1
−26
28
0.0910
9
27.27
26
13.49
10
77.02
C
2
−29
29
NA
5
10.70
12
4.20
4
46.49
U
1
−49
20
0.0139
30
83.62
16
4.82
29
440.67
U
2
−49
19
0.0124
28
82.40
17
4.83
30
540.59
Table 95: Tabulation of operationally-ranked results for Class B — Left and Right Slap. Letter refers to the participant’s letter code found on the footer
of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Score refers to the sum of the scoring
equations shown in Appendix L. The color bands are indicative of the 80th percentile, 55th percentile, and below of the Score column. The FNIR column
was computed at the score threshold that gave FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed.
The Identification column shows the time used to perform a search over an enrollment set of 3 000 000, as seen in Table 19. The Enrollment column shows
the time used to create a search template to be used for a query, as seen in Table 77. Identification and Enrollment durations are reported in seconds,
but were originally recorded to microsecond precision. RAM refers to the sum of the resident set sizes of the stage one identification processes over all
compute nodes after returning from the identification stage one initialization method, as seen in Table 63. RAM is reported in gigabytes, where 1 GB is
equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green
and the worst in pink. The table is sorted on descending Score.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Score
FNIR
215
Identification
Enrollment
0.0012
7
19.07
25
16.52
5
49.68
2
0.0012
14
42.92
20
8.92
1
7.54
2
0.0012
24
66.01
20
8.92
2
7.54
2
0.0012
18
46.70
24
11.86
13
79.43
1
0.0009
16
43.86
30
19.36
21
108.71
Letter
Sub. #
I
1
132
2
Q
1
129
Q
2
119
D
2
114
I
2
109
RAM
L
1
90
10
0.0031
5
14.50
2
3.05
22
119.79
V
2
83
7
0.0024
19
49.30
7
3.36
9
63.53
E
1
72
16
0.0043
2
6.76
13
5.34
14
79.73
G
1
71
18
0.0062
1
4.26
18
7.90
25
156.12
V
1
71
9
0.0027
10
34.96
7
3.36
8
63.53
D
1
52
6
0.0020
17
45.15
17
6.16
12
79.43
L
2
41
11
0.0033
9
28.59
1
0.88
27
177.48
O
1
39
15
0.0041
11
35.64
5
3.36
19
101.00
J
1
36
17
0.0049
8
25.38
3
3.35
18
101.00
O
2
31
13
0.0035
21
60.02
5
3.36
20
101.00
J
2
27
11
0.0033
22
60.13
3
3.35
17
101.00
E
2
20
7
0.0024
15
43.61
16
5.93
28
317.95
G
2
7
14
0.0040
6
16.26
18
7.90
26
156.12
M
1
1
26
0.0543
13
38.87
28
18.11
6
57.53
M
2
−8
25
0.0515
23
65.91
28
18.11
6
57.53
C
1
−8
30
NA
3
7.92
11
5.13
3
46.49
U
2
−17
22
0.0141
25
75.28
14
5.73
30
540.60
S
1
−18
20
0.0108
29
87.60
22
10.31
24
150.35
H
1
−19
23
0.0203
26
82.66
9
4.37
15
86.46
S
2
−20
21
0.0136
30
88.46
22
10.31
23
150.35
H
2
−21
24
0.0204
27
86.56
9
4.37
16
86.46
C
2
−28
29
NA
4
10.21
11
5.13
4
46.49
F
1
−29
27
0.0591
12
37.56
26
18.07
10
77.02
F
2
−33
27
0.0591
20
49.34
26
18.07
10
77.02
U
1
−49
19
0.0099
28
86.60
15
5.82
29
440.68
Table 96: Tabulation of operationally-ranked results for Class B — Identification Flats. Letter refers to the participant’s letter code found on the footer of
this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Score refers to the sum of the scoring
equations shown in Appendix L. The color bands are indicative of the 80th percentile, 55th percentile, and below of the Score column. The FNIR column
was computed at the score threshold that gave FPIR = 10−3 . NA indicates that the operations required to produce the value could not be performed.
The Identification column shows the time used to perform a search over an enrollment set of 3 000 000, as seen in Table 19. The Enrollment column shows
the time used to create a search template to be used for a query, as seen in Table 78. Identification and Enrollment durations are reported in seconds,
but were originally recorded to microsecond precision. RAM refers to the sum of the resident set sizes of the stage one identification processes over all
compute nodes after returning from the identification stage one initialization method, as seen in Table 63. RAM is reported in gigabytes, where 1 GB is
equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in green
and the worst in pink. The table is sorted on descending Score.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
216
FPVTE – F INGERPRINT M ATCHING
Participant
Score
FNIR
Letter
Sub. #
Q
1
133
2
Identification
Enrollment
RAM
0.0011
9
36.86
20
9.11
2
13.33
I
2
114
1
0.0010
10
38.28
30
16.99
11
110.22
D
2
108
2
0.0011
26
74.36
24
13.66
15
132.37
Q
2
102
4
0.0013
8
35.35
20
9.11
1
13.33
L
1
101
16
0.0102
3
17.51
1
3.48
23
192.19
I
1
78
4
0.0013
16
52.41
29
16.67
4
84.42
E
1
74
14
0.0088
2
12.77
11
5.20
14
127.39
J
1
66
12
0.0047
12
43.57
5
3.62
20
181.00
V
1
55
7
0.0024
14
47.02
7
3.66
13
121.80
G
1
51
23
0.0368
1
7.71
18
8.32
25
274.35
V
2
51
7
0.0024
19
62.26
7
3.66
12
121.79
L
2
39
15
0.0095
6
23.53
1
3.48
27
274.95
O
1
35
9
0.0025
23
68.54
3
3.61
19
181.00
C
2
32
24
0.0711
5
18.38
13
5.80
10
92.86
O
2
31
10
0.0027
24
73.41
3
3.61
22
181.00
H
2
23
19
0.0275
20
65.36
9
5.16
18
144.02
H
1
21
20
0.0276
21
68.07
9
5.16
17
144.02
D
1
11
6
0.0015
25
73.93
17
7.10
16
132.37
J
2
9
10
0.0027
28
77.75
6
3.62
21
181.00
G
2
3
20
0.0276
7
23.69
18
8.32
26
274.35
M
2
2
27
0.0826
13
46.70
25
15.83
8
90.93
F
1
1
25
0.0734
15
51.69
27
15.87
7
90.93
F
2
−1
25
0.0734
18
60.86
27
15.87
6
90.93
U
2
−7
17
0.0155
22
68.30
15
5.91
29
811.45
U
1
−10
18
0.0163
27
74.94
15
5.91
29
811.45
S
1
−21
22
0.0311
29
86.56
22
10.87
24
260.37
M
1
−21
28
0.0934
11
43.37
25
15.83
5
90.93
E
2
−25
13
0.0048
17
52.91
12
5.20
28
573.25
C
1
−35
30
NA
4
18.30
13
5.80
9
92.86
S
2
−94
29
0.1680
30
91.74
22
10.87
3
61.04
Table 97: Tabulation of operationally-ranked results for Class C — Ten-Finger Plain-to-Plain. Letter refers to the participant’s letter code found on the
footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Score refers to the sum of
the scoring equations shown in Appendix L. The color bands are indicative of the 80th percentile, 55th percentile, and below of the Score column. The
FNIR column was computed at the score threshold that gave FPIR = 10−3 . NA indicates that the operations required to produce the value could not
be performed. The Identification column shows the time used to perform a search over an enrollment set of 5 000 000, as seen in Table 20. The Enrollment
column shows the time used to create a search template to be used for a query, as seen in Table 79. Identification and Enrollment durations are reported in
seconds, but were originally recorded to microsecond precision. RAM refers to the sum of the resident set sizes of the stage one identification processes
over all compute nodes after returning from the identification stage one initialization method, as seen in Table 64. RAM is reported in gigabytes, where
1 GB is equal to 1 073 741 824 bytes. The number to the left of a value provides the value’s column-wise ranking, with the best performance shaded in
green and the worst in pink. The table is sorted on descending Score.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology
FPVTE – F INGERPRINT M ATCHING
Participant
Score
FNIR
217
Identification
Enrollment
0.0014
7
30.82
24
18.73
11
137.03
1
0.0013
15
54.56
25
20.23
4
113.80
2
101
2
0.0014
25
74.22
17
13.06
2
20.22
1
90
5
0.0017
10
40.74
9
8.50
17
234.03
D
1
84
4
0.0015
17
58.92
21
17.39
10
132.40
L
2
73
14
0.0083
5
20.25
3
4.36
26
367.82
E
1
71
18
0.0106
2
8.63
12
9.92
16
191.66
Q
1
69
5
0.0017
26
74.40
17
13.06
1
20.22
L
1
66
17
0.0097
8
31.68
3
4.36
21
280.49
C
2
55
15
0.0085
6
21.97
13
10.69
15
183.36
C
1
55
16
0.0094
4
20.15
13
10.69
14
183.36
Letter
Sub. #
I
2
124
2
I
1
115
Q
V
RAM
V
2
51
8
0.0019
18
65.07
9
8.50
18
234.03
G
1
49
24
0.0447
1
7.78
19
15.50
20
261.29
J
1
44
13
0.0051
9
33.02
5
6.69
22
303.13
O
1
34
11
0.0034
14
54.42
6
6.71
25
303.13
G
2
33
21
0.0333
3
19.46
19
15.50
19
261.29
O
2
32
9
0.0033
21
69.30
6
6.71
23
303.13
J
2
31
9
0.0033
19
67.98
8
6.74
24
303.13
U
2
15
22
0.0351
24
72.48
1
2.87
28
806.17
M
2
6
27
0.0716
13
48.80
26
20.89
5
130.29
E
2
4
12
0.0050
11
42.73
11
9.91
30
930.48
D
2
3
7
0.0018
27
74.97
30
30.47
9
132.37
F
1
2
25
0.0536
16
55.79
28
20.92
7
130.29
U
1
−3
23
0.0358
28
82.61
1
2.87
28
806.17
F
2
−3
25
0.0536
20
68.20
28
20.92
6
130.29
H
2
−18
19
0.0199
29
84.50
15
12.17
12
144.02
H
1
−18
20
0.0201
30
84.51
15
12.17
13
144.02
M
1
−29
28
0.0783
12
48.08
26
20.89
8
130.29
S
2
−56
30
0.2462
23
70.77
22
17.57
3
78.28
S
1
−96
29
0.0860
22
69.71
22
17.57
27
382.88
Table 98: Tabulation of operationally-ranked results for Class C — Ten-Finger Rolled-to-Rolled. Letter refers to the participant’s letter code found on
the footer of this page. Sub. # is an identifier used to differentiate between the two submissions each participant could make. Score refers to the sum of
the scoring equations shown in Appendix L. The color bands are indicative of the 80th percentile, 55th percentile, and below of the Score column. The
FNIR column was computed at the score threshold that gave FPIR = 10−3 . The Identification column shows the time used to perform a search over an
enrollment set of 5 000 000, as seen in Table 20. The Enrollment column shows the time used to create a search template to be used for a query, as seen in
Table 80. Identification and Enrollment durations are reported in seconds, but were originally recorded to microsecond precision. RAM refers to the sum
of the resident set sizes of the stage one identification processes over all compute nodes after returning from the identification stage one initialization
method, as seen in Table 65. RAM is reported in gigabytes, where 1 GB is equal to 1 073 741 824 bytes. The number to the left of a value provides the
value’s column-wise ranking, with the best performance shaded in green and the worst in pink. The table is sorted on descending Score.
C = afis team D = 3M Cogent E = Neurotechnology F = Papillon
G = Dermalog H = Hisign Bio-Info Institute
I = NEC
J = Sonda
K = Tiger IT
L = Innovatrics M = SPEX
O = ID Solutions
P = id3
Q = Morpho
S = Decatur Industries T = BIO-key
U = Aware
V = AA Technology