Single-Side Domain Generalization for Face Anti-Spoofing
- Publication
- CVPR 2020Conference
- First submitted to arXiv
- arXiv:2004.14043
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{faceantispoofingssdg2020,
title = {{Single-Side Domain Generalization for Face Anti-Spoofing}},
author = {Yunpei Jia and Jie Zhang and Shiguang Shan and Xilin Chen},
year = {2020},
eprint = {2004.14043},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2004.14043}
}
The techniques used to attack facial recognition systems are varied and plentiful, and the solutions to address them are no less innovative.
Starting with this paper, let's look at FAS research based on Domain Generalization (DG) techniques.
Problem Definition
For the FAS problem, researchers have developed two major technical routes:
-
Texture-Based: This method identifies real and fake faces through differences in image details, such as color distortion, material texture abnormalities, etc. Methods include handcrafted features (e.g., LBP, HOG, SIFT) as well as recent CNN feature learning.
-
Time-Series Based: This method analyzes the changes in multiple frames of images, such as eye blinking, mouth movements, or rPPG pulse signals, and uses CNN-LSTM or RNN architectures to capture liveness indicators.
These methods perform well within a specific dataset (intra-database), but once the model is applied to another dataset (cross-database), the accuracy drops sharply.
For example:
Let's look at the image on the left first.
We can imagine each color as a phone model. This is an analogy for domain shift, not the actual device labels in the figure:
- Green: iPhone XX
- Purple: Samsung OO
- Blue: Pixel YY
Then there's:
- Orange: A phone with imaging quality that is drastically different from others but not included in the training dataset.
After training an FAS model with the first three datasets, we discover that the pass rate for the "Orange" phone is particularly low. However, it turns out that many users of the "Orange" phone exist, so when we deploy the model, we receive a large number of complaints.
What we expected was the scenario on the right, where the model had a certain level of generalization ability during training and could perform well across different data sources. But in reality, we were slapped with the harsh truth.
The inability to generalize effectively to different data sources is a recurring issue, and there are several reasons for it:
- Distribution Differences: The training and testing datasets come from different sources, such as background, resolution, lighting conditions, capturing devices, and demographic composition.
- The diversity of fake faces is much higher than that of real faces, with variations in attack methods and production processes, making it difficult to consolidate fake face features and leading to blurred classification boundaries.
Traditional methods rely too much on dataset-specific features, resulting in models learning representations with obvious "dataset bias."
Therefore, the authors of this paper believe that the FAS proposition should be:
Design a highly generalized model without using any target data.
Otherwise, no matter how much data we collect, we will still not be able to handle fake faces from different sources and attack types.
Solution
To address the generalization difficulty caused by the "high heterogeneity of fake face features" and the "relative consistency of real faces," the authors propose a novel anti-spoofing training strategy: Single-Side Domain Generalization (SSDG).
As shown in the image above, the design philosophy of this framework can be summarized in one sentence:
Bring genuine-face features together across domains, while preserving differences between attack sources.
In other words, SSDG does not force the creation of a common generalization space for both real and fake faces, but instead adopts an asymmetric strategy—cross-domain adversarial training is applied to real faces, making their distribution tend to be consistent, while for fake faces, the distribution differences are deliberately retained and even strengthened for their specificity.
This asymmetric learning design aims to produce a classification boundary that generalizes better to unseen target datasets.
The overall model includes the following key modules:
- Feature Generator (G)
- Domain Discriminator (D)
- Face Anti-Spoofing Classifier
- Asymmetric Triplet Loss Training
- Feature and Weight Normalization Modules
Let’s take a closer look at each of these modules.
Single-Side Adversarial Learning
In the DG (Domain Generalization) field, a common technique for overcoming cross-domain challenges is to introduce adversarial training. This prevents the model from distinguishing which domain a sample originates from, allowing it to learn domain-invariant features.
The method is simple: add a "domain classifier" to the model, teaching it to classify samples as either "source A" or "source B."
The domain classifier still minimizes its classification loss to identify the source. The feature generator receives the opposite gradient and tries to increase that loss, making the source harder to identify. Better classification does not make cross-entropy larger; the two modules optimize in opposing directions.
In this paper, the authors introduce a new adversarial learning method:
- Single-Side Adversarial Learning (SSAL).
The modeling approach is as follows:
There are source datasets, denoted as , each containing:
- Real face samples
- Fake face samples
These are fed into the feature generator, resulting in feature representations:
Since parameters are shared, and are the same model , with the distinction being in the input source.
Next, the features are input to the domain classifier to determine which dataset they belong to. To enable the model to learn cross-domain consistent real face features, the feature generator attempts to "fool" the classifier, creating an adversarial learning process.
The overall loss function is:
Where is the domain label, and is the indicator function, signifying whether the sample belongs to the -th dataset.
Fake faces are excluded from the domain-adversarial branch, but still participate in genuine-versus-attack classification and asymmetric triplet learning. They are not ignored by training as a whole.
To implement this "train one side, adversarily counteract the other" mechanism, the authors use the Gradient Reversal Layer (GRL). This is a common trick used in DG models, which we’ll explain shortly.
Asymmetric Triplet Loss
Since we are abandoning the idea of compressing fake faces into a shared feature space, we need to find a way to separate them and increase the distance from real faces. This is the problem that the asymmetric triplet loss aims to solve.
The implementation works as follows:
Suppose there are three source datasets. We reclassify the samples into four categories:
- All real faces → One category (Cross)
- Fake samples from the three source datasets → One category per source (Circle / Square / Triangle)
The goal is to satisfy the following:
- Separate fake faces from different source domains while bringing fake faces from the same source closer (inter-domain separation and intra-domain compactness)
- Cluster real faces together (intra-class compactness)
- Increase the distance between real and fake faces (inter-class separation)
As shown in the image below:
For each anchor/positive/negative triplet combination , the asymmetric triplet loss is defined as:
Here is the margin and . Equation (3) in the paper omits the positive-part operator; the expression here makes the usual hinge explicit. The authors’ public HardTripletLoss also clips with ReLU and handles triplet selection and averaging. Whether distances are squared depends on the implementation setting.
This design forces the model to naturally separate real and fake faces in the feature space, while maintaining stable and compact distributions for real faces.
Feature and Weight Normalization
The authors further introduce a dual normalization design to enhance feature consistency and classification boundary stability:
-
Feature Normalization: Apply L2 normalization to all output features, ensuring they lie on the unit sphere, thereby removing feature-magnitude differences; this does not eliminate every distribution shift caused by lighting or camera conditions.
-
Weight Normalization: Normalize the classifier's weight vectors to unit length, and set biases to zero, simplifying the classification decision to:
This allows the model to focus on directional differentiation rather than relying on feature magnitude, further stabilizing the decision process.
After integrating these modules, the overall SSDG framework's training loss is:
Where:
- : The cross-entropy loss for the general classifier
- : Balancing coefficients to control the weight of adversarial and triplet losses
The combined loss must be interpreted together with GRL: the domain discriminator minimizes the adversarial term, while that term’s gradient is reversed for the generator. Both modules should not directly minimize the same positive cross-entropy in the same direction.
Compared with the staged MADDG training procedure discussed in the paper, SSDG jointly optimizes these modules.
Additional Supplement
What is GRL? Why is it so important for DG?
The Gradient Reversal Layer (GRL) is a familiar tool for adversarial feature learning in Domain Generalization (DG) and Domain Adaptation (DA). This DANN line of work dates to a 2014 preprint and the ICML 2015 paper Unsupervised Domain Adaptation by Backpropagation.
To explain GRL in one sentence:
Do nothing during the forward pass, but secretly multiply the gradient by -1 during the backward pass.
This allows us to create a tug-of-war effect between two training objectives: the feature extractor tries to make the data source indistinguishable, while the domain classifier attempts to identify the data source, which is the essence of adversarial learning.
DG aims to generalize to domains unseen during training. Learning domain-invariant features is one approach, and adversarial training is one tool for doing so:
- If the data is from different domains, the feature extractor must deceive the domain classifier so that it cannot distinguish between domains.
- The two modules compete to preserve task information while reducing source information.
GRL is the tool that implements this tug-of-war mechanism, allowing the whole process to be optimized automatically using standard backpropagation.
Here's a simple implementation of the forward and backward processes of GRL:
import torch
from torch.autograd import Function
class GradientReversalFunction(Function):
@staticmethod
def forward(ctx, x, lambd):
ctx.lambd = lambd
return x.view_as(x)
@staticmethod
def backward(ctx, grad_output):
return grad_output.neg() * ctx.lambd, None
def grad_reverse(x, lambd=1.0):
return GradientReversalFunction.apply(x, lambd)
The following example inserts GRL into the forward pass and uses real_mask to restrict the SSDG domain branch to genuine samples. It illustrates data flow, not the full SSDG training procedure. Domain loss must use only the corresponding genuine-sample domain labels, and the batch must contain genuine samples.
class DomainAdversarialNetwork(torch.nn.Module):
def __init__(self, feature_extractor, classifier, domain_discriminator):
super().__init__()
self.feature_extractor = feature_extractor
self.classifier = classifier
self.domain_discriminator = domain_discriminator
def forward(self, x, real_mask, lambd=1.0):
feat = self.feature_extractor(x)
class_out = self.classifier(feat)
reversed_feat = grad_reverse(feat[real_mask], lambd)
domain_out = self.domain_discriminator(reversed_feat)
return class_out, domain_out
In this example, lambd is a hyperparameter that controls the strength of the reverse gradient. In the SSDG paper, a dynamic adjustment formula for lambd is used:
This design allows the model to focus on the classification task during the early stages of training, and as training progresses, the intensity of adversarial learning increases, ultimately achieving a balance.
Discussion
Dataset Testing Setup
The authors conducted experiments using four publicly available datasets, sourced as follows:
- O: OULU-NPU
- C: CASIA-FASD
- I: Replay-Attack
- M: MSU-MFSD
Each dataset is held out in turn as an unseen target, while the other three serve as source training sets. This gives four fixed tasks:
- O & C & I → M
- O & M & I → C
- O & C & M → I
- I & C & M → O
This held-out design tests how the model handles unseen data sources. Actual deployment may involve shifts that these four datasets do not cover.
Comparison with Other Methods
The authors summarized the performance of the most representative FAS methods on four cross-database testing tasks, including:
- CNN-based methods (Yang et al., Atoum et al.)
- Time-series and physiological signal methods (Liu et al. rPPG)
- Traditional feature-based methods (LBP, SURF)
- MADDG (the DG baseline in this comparison table)
In the table above, if SSDG uses an MADDG-based architecture, it is called SSDG-M; if it uses a ResNet18-based architecture, it is called SSDG-R.
In this table, both SSDG-M and SSDG-R outperform the MADDG baseline on HTER and AUC across all four tasks. SSDG-R also has the lowest HTER and highest AUC among the listed methods for each task:
- Lowered HTER
- Increased AUC
- The ranking is consistent across these four tasks; repeated-run variability was not reported.
Based on these results, the authors make several key observations:
-
Most existing methods do not address the cross-database distribution issue:
- Most listed methods have higher HTER than SSDG on these four cross-database tasks.
- The authors suggest dataset-specific features as a possible reason for limited generalization, a hypothesis that needs direct feature analysis.
-
MADDG is a rare exception, but it still has blind spots in its strategy:
- It tries to build a generalized feature space for both real and fake faces.
- SSDG performs better than MADDG on these four tasks.
In contrast, the asymmetric learning strategy adopted by SSDG is more pragmatic:
- Real faces: cross-domain alignment, learning stable representations
- Fake faces: actively separating, not forcing alignment
The ablation below offers a closer check on how much the asymmetric strategy contributes.
Ablation Experiment
In this table, the authors break down the three core modules of SSDG to observe the impact of removing each module:
- ssad: Single-side adversarial learning
- triplet: Asymmetric triplet loss
- norm: Feature and weight normalization
For both architectures, removing any module increases HTER on all four tasks. AUC has an exception: for SSDG-R on O & C & M → I, removing ssad yields 96.63%, slightly above the full model’s 96.59%. The results support the combination of modules, but not a strict improvement in every metric.
Conclusion
The SSDG framework responds to an often-overlooked difference through its training strategy, rather than simply building a larger network or collecting more data:
Genuine faces may share stable cues across domains, while spoofed faces can differ greatly by attack source. They need not be treated alike.
This is a modeling assumption, not a claim that every genuine sample is stable or every attack is chaotic. In the four reported cross-database tasks, the approach performs well even with a compact architecture. What matters here is the choice to align real faces across domains without forcing every fake face into one shared cluster.
Looking back, that asymmetric objective is what makes SSDG a useful landmark in this line of FAS research. Later domain-generalization methods may take different forms, but the question it raises—what should be aligned, and what should remain distinct—has stayed with us.