Generalizable Representation Learning for Mixture Domain Face Anti-Spoofing
- Publication
- AAAI 2021Conference
- First submitted to arXiv
- arXiv:2105.02453
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{faceantispoofingd2am2021,
title = {{Generalizable Representation Learning for Mixture Domain Face Anti-Spoofing}},
author = {Zhihong Chen and Taiping Yao and Kekai Sheng and Shouhong Ding and Ying Tai and Jilin Li and Feiyue Huang and Xinyu Jin},
year = {2021},
eprint = {2105.02453},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2105.02453}
}
Earlier, we reviewed SSDG, which employs a single-side domain generalization technique to address the FAS problem.
This time, the once-popular Meta Learning has also joined the battlefield.
Defining the Problem
Following the preliminary success of DG techniques in FAS, the authors of this paper immediately raised the next question:
What exactly is a "domain"?
Is it a different shooting scene? A different attack method? Or are you just using the dataset name as the domain label?
Such definitions have often been used without much discussion in DG tasks. The authors argue that these ways of dividing domains are ad hoc and may not capture the factors that matter in mixed data.
The authors highlight three issues:
-
Manually labeling domain labels is too time-consuming.
If someone has to say, "This belongs to the domain with yellowish lighting," standardization becomes a problem.
-
The domain itself is a complex factor and cannot be divided by a single dimension.
Some domains are because of poor cameras, while others are because the model looks fake. Who can say these two should belong to the same category?
-
Even if you divide them by dataset, it is still sub-optimal.
Dataset A and Dataset B actually have a lot of overlap, and the model might have already mixed them up.
In response to this dilemma, the authors propose a solution:
Can we let the model learn to group itself?
Allow the model to generate pseudo-domains and then train its generalization ability from these pseudo-domains.
This is the core idea behind D²AM.
Solving the Problem
The model architecture diagram provided by the authors looks quite complex and might feel intimidating.
So, we need to pair it with another section in the paper to understand this architecture. Please refer to the following algorithm logic:
Stage 1: Clustering
The task in this stage is very clear:
"I don’t know which domain these images come from, but that’s okay, I’ll group them myself."
The approach is roughly as follows:
- Extract CNN style statistics (mean, std)
- Pass through the DRLM module, splitting the features into and
- Use entropy regularization to suppress task-discriminative information in
- Select the lowest-attention half of the channels in , extract their statistics, and cluster them with K-means
Let’s go through the detailed steps:
-
Step 1: Preprocessing
Refer to Algo1: Lines 2–5. Initially, the authors use a ResNet pretrained on ImageNet to process all the images.
This is not for classification but to extract style features, resulting in the following vector:
These statistics initialize the cluster count and pseudo-domains. Subsequent epochs recompute them using D²AM’s own extractor (Algorithm 1, Lines 6–7).
-
Step 2: Enhance domain features with DRLM
DRLM is a small module, similar to an SE block, that adds channel attention to CNN features:
Then, it divides the original features into two parts:
- : These are the task-related features.
- : These are the domain-related features that we care about.
-
Step 3: Discard unnecessary task information and clean with Entropy loss
The authors introduce an "anti-entropy" regularization:
where
The intended role is to suppress live/spoof information in . There is a formula inconsistency, however: with the scalar sigmoid probability defined in the paper, minimizing gives , not . Binary negative entropy, , is minimized at . We retain the paper’s Eq. (7) and flag the mismatch with its prose; the authors’ implementation has not been verified.
-
Step 4: K-means clustering to produce pseudo-domain labels (Algo1: Line 8)
Sort the channel attention weights and select the lowest channels of to form . The per-channel means and standard deviations across layers form , which is clustered rather than a flattened full feature map. Positive and negative samples are clustered separately, then matched into pseudo-domains. Genuine/attack labels are still required.
To avoid the clusters jumping around, the authors use the Kuhn-Munkres (Hungarian algorithm) to align and match epochs.
Stage 2: Meta-learning
With the pseudo domains already clustered, the model now enters the "simulation-based training" phase.
The spirit of this phase is similar to meta-learning:
"I divide the data into different small countries (domains), and continuously simulate 'What I learned in Country A, can it be applied in Country B?'"
Each epoch starts with reclustering. Each training iteration then selects meta-train and meta-test domains:
- Meta-train: Train on K−1 domains
- Meta-test: Validate generalization on the remaining domain
- Meta-optimize: Update parameters based on training and testing results
This part involves doing two things every time during training: Meta-train and Meta-test.
-
Step 5: Randomly select K−1 pseudo-domains for Meta-train (Algo1: Lines 10–12)
For each domain , a batch is extracted, and three actions are performed:
- Classification loss (with the negative sign required for minimization; the paper's Eq. (8) omits it):
- MMD regularization:
comes from the meta learner’s adaptation layer; is a same-dimensional vector sampled from a prior, a standard normal distribution in the experiments. This does not use the genuinely unseen target dataset for adaptation. MMD regularizes features toward this prior.
- Depth regression (PRNet pseudo-depth for genuine images, zero targets for attacks):
-
Step 6: Perform Meta-test (Algo1: Lines 15–16)
The meta-test domain is still part of the training data, held out only relative to this inner update. It is not the final unseen target dataset. A batch is evaluated by each inner-updated meta learner:
Finally, the depth loss is also included.
-
Step 7: Meta-Optimization (Algo1: Line 17)
The parameters of the three modules are updated together, corresponding to Eq. 11–13:
- : Feature extractor
- : Meta learner
- : Depth estimator
Each update also includes the entropy loss from DRLM:
The aim is to keep learning domain groupings that make generalization training more challenging.
Finally, let’s summarize how this pipeline works:
- CNN feature → DRLM: Split into and
- Extract → Clustering to produce pseudo domain labels
- Use these labels as the training basis for meta-learning
- In the next epoch, extract features again → Re-cluster → Train again
- Repeat until training ends, then evaluate generalization on an unseen target dataset.
Overall, this is a "self-classifying, self-training, self-disciplining" algorithm that constantly improves itself.
Discussion
Comparison with Other Methods
After all this discussion, is D²AM really stronger?
The authors provide a series of experimental results in the table above, using four mainstream FAS datasets as "unknown test sets," allowing the model to challenge generalization capabilities with only three source domains.
D²AM records the lowest HTER in three of the four target-dataset settings and the highest AUC in two. With CASIA as target (O&M&I → C), RFM has both lower HTER (20.27% versus 20.98%) and higher AUC (88.16% versus 85.58%). With OULU as target, D²AM has lower HTER, but RFM has higher AUC (91.16% versus 90.87%).
On the other three target datasets, D²AM has lower HTER. It builds its training pseudo-domains by clustering without source-dataset domain labels, although live/spoof labels are still needed. Run-to-run stability requires repeated training and evaluation.
t-SNE Visualization
The authors thoughtfully used t-SNE to create several plots to help with understanding:
- (task features): shows more live/spoof class information.
- (domain features): shows more source/style information, without establishing that all task information is absent.
In other words, helps clarify "which domain the sample is from," while handles "whether the sample is real or fake."
These plots are consistent with DRLM’s intended behavior, but two-dimensional t-SNE visualizations are qualitative evidence.
Clustering Visualization
The authors further observed the behavior of pseudo-domains and made some interesting discoveries:
-
As epochs progress, the basis for D²AM's clustering changes:
- In Epoch 3, the focus is on lighting.
- In Epoch 6, the focus shifts to the background.
-
The normalized mutual information (NMI) between neighboring epochs is around 0.6–0.8, showing that the cluster labels keep changing during training.
The paper reports 64.9% ± 5.7% NMI between pseudo-domain labels and domains defined by source datasets. This measures similarity between label partitions, not the percentage of samples that "overlap."
The clustered partition is therefore different from a direct dataset-based split. Which partition works better still depends on downstream evaluation.
Conclusion
D²AM proposes a completely different approach in the domain generalization (DG) field.
I don’t need you to label the domains for me; I will group them, train on them, and get better on my own.
Its core idea is simple: "Generalization should not rely on knowing the domain in advance."
Through DRLM, entropy loss, and MMD regularization, D²AM learns a more challenging way to partition data without domain labels, then trains an FAS model on those partitions. Its reported results are promising across several settings. The paper's printed entropy equation also needs care, as noted above.
This is more than a single FAS example. For data drawn from mixed sources with unknown domains, D²AM offers a thought-provoking starting point for machine learning more generally.