In 2005 through 2006 we carried out a study of factors that influenced the performance of three algorithms from the Face Recogntion Grand Challenge. The results of this study are summarized in the following paper that is currently under review by Computer Vision and Image Understanding. For the moment there is a direct link here, and should the manuscript be accepted, this link and citation will be modified appropriately.
Factors that Influence Algorithm Performance in the Face Recognition Grand Challenge, J. Ross Beveridge, Geof H. Givens, P. Jonathon Phillips and Bruce A. Draper, Under review by Computer Vision and Image Understanding, (to appear).
For the sake of providing a detailed record of the data that went into our analysis, the three algorithm performance data in exactly the tabulated format used as input to our study is available here. Some additional comments about the exact choice of data in this file appear below.
We also want to acknowledge Vijayakumar Bhagavatula and Marios Vavvides at CMU as well as Chengjun Liu at the NJIT for permission to distribute data based upon the performance of their algorithms in the Face Recognition Grand Challenge. Further, we should add that the performance of the CMU algorithm reflects a submission prior to the final and hence absolute performance levels are below what is reported for the final submission.
More Detail on the Tabulated Data
The list of subjects was chosen from those included in the FRGC experiment 4. For a subject to be included in our study, we required there to be at least 8 query images and 16 target images. For subjects with more imagery, 16 target and 8 query images were selected at random. This process left us with 351 out of the 466 subjects in experiment 4, and 128 match pairs per subject. Initially that gave us 135,168 match pairs to study. However, for the following reasons, some match pairs were dropped.
All 384 outcomes for subject 4542 were deleted because the subject wore glasses in some of the controlled lighting imagery, and was the only subject to do so. The number 384 is a consequence of there being 128 outcomes for each of 3 algorithms. Another 24 outcomes associated with image '02463d524.jpg' were deleted due to an error in how the novelty covariate was computed for this image. These deletions took us from 135,168 to 134,760 outcomes. One correction was also made. The original Notre Dame data incorrectly flagged the subject as wearing glasses in image '04748d35.jpg'.
After these cases were deleted, the final data set contained 134,760 match pairs.
