Validating a Biometric Authentication System: Sample Size Requirements

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 28; no. 12; pp. 1902 - 1319
• Main Authors: Dass, S.C., Yongfang Zhu, Jain, A.K.
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01-12-2006; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Artificial Intelligence; Authentication; Biometric authentication; Biometrics; Biometry - methods; bootstrap; Computer science; control theory; systems; Computer Simulation; Computer systems and distributed systems. User interface; Confidence; Confidence intervals; Data Interpretation, Statistical; Dermatoglyphics; Error analysis; error estimation; Exact sciences and technology; Face; Fingerprint recognition; Fingerprints; Gaussian copula models; Humans; Iris; Models, Statistical; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; ROC confidence bands; Sample Size; Samples; Software; Statistical analysis; Statistical methods; Studies; System performance; Testing; Biometrics; Performance evaluation; Validation; Estimation error; Fingerprint; Statistical analysis; Gaussian copula models; Error estimation; Sample size; Biometric authentication; bootstrap; Modeling; ROC confidence bands; Bootstrapping; Gaussian process; Confidence; User interface; Facies; Pattern analysis; Artificial intelligence
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2006.255

Authentication systems based on biometric features (e.g., fingerprint impressions, iris scans, human face images, etc.) are increasingly gaining widespread use and popularity. Often, vendors and owners of these commercial biometric systems claim impressive performance that is estimated based on some proprietary data. In such situations, there is a need to independently validate the claimed performance levels. System performance is typically evaluated by collecting biometric templates from n different subjects, and for convenience, acquiring multiple instances of the biometric for each of the n subjects. Very little work has been done in 1) constructing confidence regions based on the ROC curve for validating the claimed performance levels and 2) determining the required number of biometric samples needed to establish confidence regions of prespecified width for the ROC curve. To simplify the analysis that addresses these two problems, several previous studies have assumed that multiple acquisitions of the biometric entity are statistically independent. This assumption is too restrictive and is generally not valid. We have developed a validation technique based on multivariate copula models for correlated biometric acquisitions. Based on the same model, we also determine the minimum number of samples required to achieve confidence bands of desired width for the ROC curve. We illustrate the estimation of the confidence bands as well as the required number of biometric samples using a fingerprint matching system that is applied on samples collected from a small population