Biometric Face Presentation Attack Detection with Multi-Channel Convolutional Neural Network
- Publication
- IEEE Transactions on Information Forensics and Security 2020Journal
- First submitted to arXiv
- arXiv:1909.08848
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{faceantispoofingwmca2019,
title = {{Biometric Face Presentation Attack Detection with Multi-Channel Convolutional Neural Network}},
author = {Anjith George and Zohreh Mostaani and David Geissenbuhler and Olegs Nikisins and Andre Anjos and Sebastien Marcel},
year = {2019},
eprint = {1909.08848},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1909.08848}
}
In the development of a field, new terms are often defined. These terms not only describe the technology but also influence the direction of future research.
The paper follows the established term Presentation Attack (PA). Its explanation can be paraphrased as:
Presentation of biometric characteristics or artifacts to the capture subsystem with the aim of interfering with the biometric system, for purposes such as:
- Impersonation Attack
- Obfuscation Attack
Both situations are classified as Presentation Attack (PA).
Defining the Problem
When Convolutional Neural Networks (CNNs) entered the Face Anti-Spoofing (FAS) field, they quickly displaced many traditional handcrafted approaches. However, even with CNNs, many methods still use only RGB images for judgment: facial photos taken under ordinary visible light.
The issue is that spoofing techniques have long since evolved beyond the "Print" level. Nowadays, silicone masks, 3D models, and other sophisticated techniques have become more common. Methods relying only on RGB can encounter cases that are particularly difficult to distinguish.
As a result, some research started to introduce depth information to help, such as using Time-of-Flight (TOF), structured light, or dual-lens systems to create 3D vision, detecting facial distance and contour changes.
Since we're already introducing more dimensions to solve the problem, why not go a step further?
Besides RGB images, we can find additional information such as depth, heat, and more.
This paper starts from this perspective. The authors not only used color and depth data but also incorporated Near-Infrared (NIR) and thermal imaging (Thermal) to include more "invisible facial cues" into the model.
However, this also encounters a very practical issue: datasets are not keeping up. Most open datasets only include RGB data, and those with depth data are limited in quantity, which hinders the training of stable models, let alone performing generalization tests.
Thus, another key contribution of this paper is the introduction of a new dataset:
WMCA (Wide Multi-Channel Presentation Attack).
It not only includes various 2D and 3D spoofing techniques (such as paper masks, silicone masks, rigid materials, transparent shields, etc.), but also provides synchronized data from four channels: RGB, depth, NIR, and thermal images.
By allowing the model to see more dimensions of information, it becomes harder for attackers to deceive the system simultaneously.
Solving the Problem
Model Architecture
In terms of model design, the authors propose a multi-channel convolutional neural network architecture (Multi-Channel CNN, MC-CNN), aimed at integrating information from different sensor channels to improve the model's ability to detect spoofing attacks.
In the FAS field, the lack of datasets remains a significant issue. As a result, many studies avoid training a CNN from scratch and use pre-trained networks for transfer learning instead.
This paper follows the same approach by using a pre-trained face recognition model as the backbone. The chosen model is the lightweight LightCNN, which is used to extract facial features from each channel.
Before entering the network, images from each channel undergo alignment and normalization processing. Face detection is done using MTCNN, and feature point detection is done using SDM. The eyes and mouth are aligned to a common coordinate, and then adjusted to 128×128 grayscale images.
The RGB channel serves as the reference, and other channels, such as depth, near-infrared, and thermal images, are synchronized based on the RGB feature points to ensure consistency across both space and time. Non-RGB images are also normalized using MAD (Mean Absolute Deviation in this paper) and converted into 8-bit format for easier input into the model.
Each channel has a LightCNN branch producing a 256-dimensional feature vector. Selected lower layers, the Domain Specific Units, are adapted separately for non-RGB channels, while the higher-level pre-trained weights are shared and frozen. The RGB branch is not adapted at this stage. The four vectors are concatenated into a 1024-dimensional representation and passed to two fully connected layers for classification:
- The first fully connected layer has 10 nodes.
- The second layer is a sigmoid output (1 node) predicting the probability of a bona fide presentation.
The overall training uses binary cross-entropy (BCE) as the loss function:
Where is the true label (1 for REAL, 0 for SPOOF), and is the model's predicted probability.
WMCA Dataset
To validate the performance of multi-channel models in anti-spoofing tasks, the authors additionally created a new dataset called:
- WMCA (Wide Multi-Channel Presentation Attack).
This dataset is carefully designed not only to fill the gap in the number of channels present in existing PAD datasets but also to cover a rich and challenging variety of attack methods.
The dataset includes video data from 72 subjects, covering bona fide presentations and multiple presentation attack scenarios, with each video lasting approximately 10 seconds. The recording process uses multiple sensors synchronized to capture the data, resulting in a total of about 5.1 TB of uncompressed data, making it one of the few high-quality FAS datasets available.
WMCA uses two sets of sensors for synchronized recording, covering various spectra and depth information. The devices used are shown in the image above, where:
- (b) Seek Thermal Compact PRO: Provides thermal imaging (Thermal)
- (c) Intel RealSense SR300: Provides RGB, depth, and near-infrared (NIR) channels
These sensors are modularly installed on the same optical frame to maintain consistent shooting angles and distances. The RGB channel records at 30fps, while thermal imaging records at 15fps. Each video stores approximately 300 frames per channel (thermal images store 150 frames), with all data being uncompressed for high-quality analysis. The resulting image samples are shown below:
To ensure spatial and temporal consistency of the multi-channel data, the authors developed a custom calibration board with thermal contrast characteristics and used high-power lighting for synchronized shooting. They also used custom software to automatically capture corresponding points for calibration.
The data collection lasted for five months and spanned seven different recording stages. Each stage involved changes in lighting, backgrounds, and device settings, simulating a variety of real-world scenarios, including:
- Lighting: Sunlight, LED side lighting, office lighting, etc.
- Background: Uniform backgrounds vs. complex backgrounds
- Wearing Glasses: For bonafide samples, both "with glasses" and "without glasses" versions were recorded
- Masks and Fake Heads: The masks and mannequin heads were warmed with a blower before recording to enhance the camouflage realism in the thermal channel
All participants faced the camera with a neutral expression and maintained a distance of approximately 40 cm. The entire shooting process was controlled by an operator to ensure consistency and comparability across all data.
Seven Types of Attacks
The table above gives the number of WMCA presentations. It contains 1332 attack recordings, grouped into seven main types, each with different challenges:
- Glasses: Funny fake glasses, paper glasses, etc. (Partial occlusion attack)
- Fake Head: Various model heads, some heated to simulate body temperature
- Print Attack: Using high-resolution inkjet printers or office laser printers to output facial photos
- Replay Attack: Using an iPad to display electronic photos and replay recorded facial videos
- Rigid Mask: High-fidelity rigid masks, decorative plastic masks
- Flexible Mask: Homemade silicone masks that mimic real facial features
- Paper Mask: Paper masks designed to resemble a real identity
These attack types range from the simplest 2D attacks to highly realistic 3D attacks. Coupled with multi-spectral data, this significantly enhances the dataset’s utility for training and testing models’ generalization abilities.
Evaluation Protocol and Experimental Setup
WMCA supports two experimental protocols corresponding to the "known attacks" and "unknown attacks" scenarios:
-
Grandtest Protocol: The simulated attack types are known. The data is divided into train / dev / eval sets, with disjoint subject identities. Each particular attack instrument occurs in only one partition, while attack categories are distributed across all three.
-
Unseen Attack Protocol (LOO): A stricter test designed for generalization ability. Seven leave-one-out (LOO) protocols are defined, where each time one attack type is excluded as an "unknown attack" and only appears in the test set. For example, in the "LOO fakehead" protocol, fake head attacks do not appear in the training or validation sets and the test set contains only bona fide presentations and that held-out attack category.
Each video is sampled for 50 frames (uniform temporal sampling), and each corresponding set of four-channel images from the same time point forms a training sample, treated as an independent sample for training and inference.
Evaluation Metrics
Evaluation uses APCER and BPCER from ISO/IEC 30107-3 and summarizes them with their average, ACER. All three should not be described as independent metrics defined by the standard:
- APCER (Attack Presentation Classification Error Rate) - The rate of attack samples being misclassified as real.
- BPCER (Bonafide Presentation Classification Error Rate) - The rate of real samples being misclassified as attacks.
- ACER (Average Classification Error Rate) - The average of the above two errors.
Additionally, ROC curves are plotted to assist in comparing the overall classification abilities of various methods.
Discussion
Baseline System Performance in Grandtest
The above chart shows the performance of the baseline system under the grandtest protocol, with thresholds based on a BPCER = 1% setting from the dev set. We can observe several key points from the table:
- Among the single-channel baselines, Infrared with RDWT-Haralick features has the lowest test ACER at 6.26%.
- FASNet uses only RGB and reaches 11.44% test ACER, below the two handcrafted Color baselines in the table.
- Score-level fusion brings the two handcrafted baseline systems to 7.54% and 3.44% test ACER. Both remain above the MC-CNN result of 0.3% in the next table.
MC-CNN vs Best Baseline (Grandtest)
The above figure compares the final performance of MC-CNN with the best baseline system on the dev and test sets:
- MC-CNN achieved ACER = 0.3% on the test set, below the two fusion baselines at 3.44% and 7.54%.
By learning joint representations across multiple channels, the model effectively integrates complementary information from RGB, Depth, IR, and Thermal channels. For the methods and dataset setting compared here, joint representations are more effective than these two score-fusion baselines.
Recognition Performance in Unseen Attack Scenarios
To evaluate the model's generalization ability to unseen attacks, the authors designed seven LOO protocols in WMCA, where one attack type is excluded from the training and validation sets and appears only in the test set:
In most unseen attack scenarios, MC-CNN has lower ACER than the listed fusion baselines. When flexible masks or replay attacks are held out, its ACER is 2.52% and 0.12%, respectively. For print attacks, it ties RDWT-Haralick fusion at 0%.
However, glasses attacks are difficult for both the baselines and MC-CNN: the latter has 84.27% APCER and 42.14% ACER. The paper points out that these partial attacks look similar across several channels to bona fide subjects wearing ordinary glasses, making them easy to mistake for genuine faces.
Conclusion
The shortcomings of this paper are quite apparent. In real-world applications, we often only have access to RGB images. The hardware costs and synchronization requirements of multi-channel sensor devices make this approach difficult to implement directly in practice.
However, from the perspective of the dataset, the paper establishes unseen-attack protocols for WMCA, continuing earlier research on unknown attacks. This is still valuable in the context of today's open-set and anomaly detection discussions. Additionally, the challenges identified by the authors (such as detecting partial attacks and the fuzzy boundaries between attack types) remain significant issues that researchers still face.
Lastly, the concept of "multi-channel" could be further expanded into "multi-modal":
Not only different spectral or depth dimensions of image inputs but also combining voice, behavior, contextual information, and other heterogeneous perception sources to develop a truly cross-perceptual-domain defense system.
From this perspective, while this paper may not be a pioneer in multi-modal AI, it provides an early prototype of sensor fusion for joint perception, which remains insightful for future developments.