Dual Spoof Disentanglement Generation for Face Anti-spoofing with Depth Uncertainty Learning
- Publication
- IEEE Transactions on Circuits and Systems for Video Technology 2022Journal
- First submitted to arXiv
- arXiv:2112.00568
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{faceantispoofingdsdg2021,
title = {{Dual Spoof Disentanglement Generation for Face Anti-spoofing with Depth Uncertainty Learning}},
author = {Hangtong Wu and Dan Zen and Yibo Hu and Hailin Shi and Tao Mei},
year = {2021},
eprint = {2112.00568},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2112.00568}
}
Some illusions have never appeared.
Not before you recognized them.
Problem Definition
When face recognition systems begin to bear responsibility for real-world verification, accuracy is no longer just a number on a leaderboard, but a boundary of risk.
Face Anti-Spoofing (FAS), as the gatekeeper of this boundary, has in recent years evolved from a binary classification task into a contest of generalization across diverse conditions.
However, most existing methods remain trapped by an unresolved assumption:
The limitation of data leads to models generally lacking the ability to recognize unknown identities and attack types.
The training datasets we use mostly come from closed environments, fixed devices, few identities, and limited attack types.
For example, the OULU-NPU and SiW training splits have only 20 and 90 subjects. Such data distributions cannot cover the range of devices, populations, complex lighting, and attack techniques that models may meet in actual deployment. The result is that a model can overfit its training samples and fail to recognize attacks under unseen conditions.
Searching for more and more data has become the default solution.
If we consider image translation methods, most can only "swap faces but not identities," so generated samples remain bound to the original identity; synthetic processes based on external data collection face practical costs such as copyright, annotation, manpower, and delay.
Furthermore, when we attempt to introduce generative models to solve the data bottleneck, we inevitably encounter another problem:
The quality of synthetic samples varies, which may disrupt the stability of supervisory signals.
Especially in a framework supervised by depth maps, blurred and distorted images can undermine accurate depth prediction, affecting the stability and interpretability of the training process.
This is a dual-problem structure:
- How to generate synthetic images that simultaneously possess new identities and existing attack patterns, improving intrinsic data variability?
- How to ensure these generated samples do not negatively pollute depth-supervised training, causing the main network to learn incorrect visual cues?
The authors propose a response on two fronts: how synthetic data are generated, and how the model is trained with them.
Before moving on to the next section, let us refocus on the original question:
How many illusions must a truly useful FAS system have seen to confidently say, "This is not real"?
Solution
To address the two major problems of "data scarcity" and "image distortion," the authors integrate two ideas: "generative data augmentation" and "uncertainty learning," establishing a dual-module architecture:
- DSDG (Dual Spoof Disentanglement Generation)
- DUL (Depth Uncertainty Learning)
Dual Spoof Disentanglement Generation (DSDG)
DSDG aims to learn a generator using the existing training data alone, then synthesize varied, paired live and attack images.
The authors use a dual-encoder Variational Autoencoder (VAE) to disentangle facial images into "identity features" and "attack features," then recombine them in the decoder to generate new paired images.
This approach simultaneously expands the coverage of identities and attack modes.
As shown in Figure (a), the overall structure consists of two encoders and one decoder, plus a disentanglement module that distinguishes spoof and identity features. The specific steps are:
-
Dual Encoder Feature Outputs
- inputs the attack image , separately extracting the "attack pattern feature" and the "identity feature" .
- maps the live image to the identity feature ;
Using the reparameterization trick, each encoder outputs the mean and standard deviation , then obtains features by
-
Reconstruction and Disentanglement
The decoder receives the three features and generates reconstructed and respectively.
To ensure the attack feature is correctly extracted, a spoof classifier is attached on with a cross-entropy loss () to enforce correct attack-type prediction.
The authors also introduce an angular orthogonality constraint () between the attack feature and identity feature , encouraging their vectors to be perpendicular rather than confused with one another.
-
Distribution Alignment and Reconstruction Loss
To achieve VAE feature distribution learning, the authors apply Kullback-Leibler (KL) divergence () to force the encoded distributions toward a standard Gaussian.
uses L1 distance between each reconstructed genuine/attack image and its input.
MMD () aligns the distributions of the two latent identity representations, while compares the reconstructed pair's identity features through a pretrained LightCNN to promote identity consistency.
All losses are combined into the total objective function:
After training the VAE-like structure, as shown in Figure (b), multiple pairs can be generated by sampling from a standard Gaussian noise. Then is copied as (so the pair shares an identity input) and passed to the decoder to generate new samples .
This generation step needs no external data, although training the DSDG spoof classifier still uses attack-type labels from the original data. The generated samples can show new identities and attack patterns learned from that data.
However, VAEs can produce blurry or locally distorted images. Adding these noisy samples directly to training may confuse depth supervision.
To address this, the authors propose Depth Uncertainty Learning as a remedy.
Depth Uncertainty Learning (DUL)
Beyond the diversity of synthetic samples themselves, the authors further address the interference problem in depth supervision by introducing an uncertainty representation mechanism that quantifies and suppresses the negative effects of generative noise via Gaussian distributions.
Traditional depth-supervised methods produce a fixed depth value for each image region. When a synthetic image has local distortions, however, its depth supervision may be unreliable.
The authors treat the depth value as a Gaussian distribution , where is the learnable depth mean and represents uncertainty.
During training, the value actually involved in the loss calculation is a sampled depth , where .
This "reparameterization" approach allows uncertainty to be learned through backpropagation.
Within this framework, the model applies two convolutional operations to the depth feature map: one outputs , the other .
KL divergence regularizes toward , whose mean is the supervisory depth value and whose variance is one. This is distributional regularization, not an explicit reduction of the loss weight for each uncertain pixel.
The authors attach the module to depth-supervised networks including ResNet, MobileNetV2, DepthNet, and CDCN, yielding pixel-level mean and standard-deviation maps; inference uses the mean depth map.
The authors use mean squared error () as the main depth supervision and add a weighted term in the total loss to control the strength of uncertainty regularization.
Overall, the total loss can be expressed as:
Here controls the weight of the generated-data loss in backpropagation. A separate parameter controls the fraction of original samples in each batch.
Through these two modules, the authors combine generated-data augmentation and depth uncertainty learning in the FAS training process.
Discussion
Through ablations, within-dataset and cross-dataset experiments, and visualizations, the authors assess DSDG's data augmentation and DUM's depth uncertainty learning. The results of each protocol still need to be read separately.
Ablation Studies
The authors ablate key hyperparameters and modules, including the effect of the number of identities, the generated-image ratio (through ), and the contribution of losses such as , , and to identity disentanglement.
-
Number of Identities and Generalization Performance
On OULU-NPU Protocol 1 (P1), as training identities increase from 5 to 20, this method's ACER falls from 5.1% to 0.3%; the CDCN baseline also falls from 8.3% to 1.0%. Both are affected by the number of training identities, and this method is lower at every count shown.
-
Synthetic Image Ratio
The authors vary , the fraction of original samples in a training batch. On OULU-NPU P1, gives 0.3% ACER, the lowest in the table.
Increasing the number of generated image pairs from 10k to 30k gives the same 0.3% ACER at 20k, 25k, and 30k. The authors subsequently use 20k pairs; the table does not show that more samples add noise.
-
Importance of Identity Disentanglement Losses
On OULU-NPU P1, the complete method has 0.3% ACER. Removing , , or separately gives 1.0%, 0.7%, and 0.7%, respectively.
These results support a contribution from all three losses under this protocol.
-
Hyperparameters and
The authors test sensitivity to the KL weight () and generated-data loss weight (). Most tested settings beat the CDCN baseline of 1.0% ACER, but gives 1.4%; the method is therefore not uniformly insensitive to these parameters.
Intra-Dataset Testing
To evaluate recognition ability within the same dataset, the authors select OULU-NPU and SiW.
- OULU-NPU (Table X): ACER across the four protocols is 0.3%, 1.2%, 1.4%, and 2.3%, the lowest values shown in the table. Protocol 4 covers changes in environment, attack, and input device.
- SiW (Table XI): ACER on the first two protocols is 0%, tied with several methods in the table. Protocol 3 reaches 3.80% ± 1.44%, higher than CDCN's 1.71% ± 0.11%.
Together, the results support good performance across OULU-NPU's four protocols, but SiW Protocol 3 is a clear exception. Stability across every attack type cannot be inferred.
Model Analysis and Visualization
The authors use various visualization methods to show the nature of generated images, attack feature disentanglement, and depth uncertainty distribution, helping readers understand DSDG and DUM’s internal workings.
-
Generated Images
The figure places original data beside generated live/attack pairs for different identities and attack patterns. It illustrates variety among the generated images, but cannot by itself verify identity consistency.
On the attack-diverse SiW-M dataset, the authors also show generated Replay, Print, 3D Mask, Makeup, and Partial Attack examples. These images cannot quantify the amount of distortion.
-
Attack Pattern Disentanglement
The image above is the t-SNE projection in Figure 7 of the paper. Without the attack-type classifier and orthogonality loss (), the attack representations are more mixed; after training the complete DSDG, the projection shows clearer clusters for the four classes. Both constraints change together in this comparison, so the figure cannot isolate the effect of either one.
-
Depth Uncertainty
This image corresponds to Figure 9 in the paper. It compares depth maps from CDCN and the method with DUM, alongside predicted standard deviations. The authors note higher standard deviation around reflections, face edges, and the background. These are qualitative examples; the image cannot show that the model directly reduces the loss weights of these pixels.
Conclusion
DSDG answers a simple but persistent question:
Without extra annotations or external data, how can we expand identity and attack diversity while limiting the damage synthetic noise does to depth supervision?
When this paper appeared, the full surge of deepfakes and the widespread use of diffusion models still lay ahead. The authors had already realized that "data problems cannot be solved by data alone," and tried to advance the boundary of generalization through two modular strategies: generation and uncertainty.
Of course, limitations remain:
- Using VAE as the generative backbone restricts resolution and detail stability;
- Dependence on depth limits extension to non-depth-supervised scenarios;
- Dependence on spoof types remains and is not fully relaxed, impacting purely unsupervised settings.
Nevertheless, these do not diminish its role in the technology chain. DSDG reminds us:
Data diversity, supervision reliability, and model robustness must be considered simultaneously, or reinforcement only stacks noise.
Models evolve, attacks mutate, and algorithms are forgotten. But those who tried to take the problem apart, rather than chase scores alone, leave traces that become footholds for those who follow.