Back to paper library
ICCV 2023Face anti-spoofingPaper notes

UDG-FAS

Fragments of Style

Original paper

Towards Unsupervised Domain Generalization for Face Anti-Spoofing

Paper authors

Yuchen Liu · Yabo Chen · Mengran Gou …

Show all 7 authors

Yuchen Liu · Yabo Chen · Mengran Gou · Chun-Ting Huang · Yaoming Wang · Wenrui Dai · Hongkai Xiong

Publication
ICCV 2023Conference
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@inproceedings{faceantispoofingudgfas2023,
  title = {{Towards Unsupervised Domain Generalization for Face Anti-Spoofing}},
  author = {Yuchen Liu and Yabo Chen and Mengran Gou and Chun-Ting Huang and Yaoming Wang and Wenrui Dai and Hongkai Xiong},
  year = {2023},
  booktitle = {ICCV},
  url = {https://openaccess.thecvf.com/content/ICCV2023/html/Liu_Towards_Unsupervised_Domain_Generalization_for_Face_Anti-Spoofing_ICCV_2023_paper.html}
}
On this page

If you think training an FAS model is already difficult, try doing it without any labeled data and let the model figure things out by itself. The situation goes from bad to incomprehensible.

That is the scene we face here.

Defining the Problem

Face Anti-Spoofing (FAS) has always been seen as the core defense line for face recognition systems, but it has two fatal flaws:

  • The high cost of data collection and annotation;
  • The model is easily invalidated when the scene changes.

If you've followed along with us through dozens of papers, you might already be familiar with these issues.

In order to maintain stability across domains (cross-dataset), various Domain Generalization methods have been proposed in the past, but they all come with a prerequisite: you need sufficient labeled data.

In reality, you might have access to a large amount of data, but no manpower or resources to annotate it.

Thus, the shift towards "unsupervised learning" began, aiming to address the issue of "lack of annotations."

Two familiar camps in contrastive learning are MoCo and SimCLR.

While these contrastive learning methods have achieved outstanding results in general image tasks, they fail to deliver as expected in FAS. When it becomes a "Real / Fake" binary task, negative samples are hard to define, and the model often fails to determine which side to reject.

BYOL and SimSiam try to remove the reliance on negative samples, but the face data in this study can still steer them toward identity and domain shortcuts. You may think the model has learned to spot a spoofed face when it has merely remembered which dataset photographed which person.

More awkward still, many methods assume that an ImageNet-pretrained backbone is available. ImageNet can teach useful visual features, but it was trained to tell “cats” from “dogs,” not real faces from spoofed ones. Under the paper's zero-shot or few-shot FAS settings, that distinction still has to be learned.

So, the problem comes back to square one:

  • If there are no annotations and no suitable pre-trained data, what can the model learn from?
  • If identity and domain noise dominate over spoofing itself, how can we make the model focus on "what it should be looking at"?

The answer proposed by this paper is: break the face apart first!

Solving the Problem

The overall design of UDG-FAS revolves around one goal: during unlabeled pretraining, gradually reduce identity and scene bias so the model can focus more on FAS cues.

This goal is broken down into three strategic modules:

  1. Reducing identity bias within a single image through Split-Rotation-Merge (SRM);
  2. Loosening cross-identity representation differences through In-domain Nearest Neighbor (NN) search;
  3. Handling style shifts between data sources via Cross-domain Nearest Neighbor (NN) search.

As always, the situation is complex, so let's go through them one by one.

SRM Module

Split-Rotation-Merge (SRM) is the key module for reducing identity bias. It aims to preserve local spoof cues while weakening dependence on facial layout and individual differences.

The overall process is as follows:

First, the input image x1x_1 undergoes non-destructive data augmentation (e.g., brightness adjustment, mirroring, etc.) and is then split into m×mm \times m local patches. Each patch x1px^p_1 undergoes random rotation, thereby breaking the symmetry and spatial arrangement bias introduced by the original face structure, and reducing the model's sensitivity to identity-related signals.

Next, these local patches are passed through an encoder ff to generate corresponding feature vectors:

e1p=f(x1p)e^p_1 = f(x^p_1)

The method enumerates every subset sis_i of nn patches among the m2m^2 patches, yielding (m2n)\binom{m^2}{n} merged vectors rather than one random subset. The equation below describes one subset; m=2,n=2m=2,n=2 gives six combinations per image:

v1=1n∑p∈se1pv_1 = \frac{1}{n} \sum_{p \in s} e^p_1

Here ss is one patch-index subset. Spoof cues can appear in several regions, while identity features vary across parts of the face. Averaging the chosen local features is intended to lessen the identity signal.

The generated local representation v1v_1 is then passed through a projector gg and predictor qq, producing a contrastive vector p1p_1.

In contrast, another image view x2x_2 is not split or rotated and is directly passed through the encoder ff and projector gg, resulting in a global feature vector z2z_2.

For the loss design, the authors adopt SimSiam’s cosine similarity loss, along with a stop-gradient mechanism to avoid collapse and stabilize training:

LSRMi=12⋅D(p1i,stopgrad(z2))+12⋅D(p2i,stopgrad(z1))\mathcal{L}_{\text{SRM}}^i = \frac{1}{2} \cdot \mathcal{D}(p_1^i, \texttt{stopgrad}(z_2)) + \frac{1}{2} \cdot \mathcal{D}(p_2^i, \texttt{stopgrad}(z_1))

D(p,z)=−p⊤z∥p∥2∥z∥2\mathcal{D}(p,z)=-\frac{p^\top z}{\|p\|_2\|z\|_2} is negative cosine similarity. Here ii indexes patch subsets; the views swap roles, and losses are averaged over all subsets. Stop-gradient belongs to the SimSiam design.

Overall, SRM is more than an ordinary augmentation: it transforms local features to reduce dependence on the full facial layout while trying to retain spoof information. Whether it achieves that aim must be judged from the experiments.

In-domain Nearest Neighbors

After handling identity bias in a single image, the next challenge UDG-FAS faces is how to further eliminate cross-identity feature diffusion across a multi-sample dataset.

Although the SRM module can effectively suppress identity signals from facial structure within a single image, in practice, the FAS task deals with datasets consisting of many cross-identity samples. Even if the samples come from the same domain, the features learned by the model can still be fragmented or stretched due to identity differences, making semantically consistent samples fail to be treated as positive examples.

To further address this issue of "cross-identity but same class" feature diffusion, the authors designed the In-domain Nearest Neighbor (IDNN) module as a complementary contrastive mechanism.

The core idea is: without relying on annotations, use the geometric proximity in the embedding space to infer sample pairs that may have similar semantics and include them in contrastive learning to enhance intra-class consistency.

Use the unaugmented image xx and feature zz to search an unaugmented in-domain support set, excluding the query itself. First obtain index k∗k^*, then retrieve the corresponding augmented-view feature for the loss. An arg⁡min⁡\arg\min returns an index, not an embedding:

k∗=arg⁡min⁡k ∥z−zkqin∥2,zNN,a=Qzain[k∗],a∈{1,2}k^*=\underset{k}{\arg\min}\,\|z-z_k^{q_{\mathrm{in}}}\|_2,\qquad z_{\mathrm{NN},a}=Q_{z_a}^{\mathrm{in}}[k^*],\quad a\in\{1,2\}

Note that the search here is limited to in-domain, meaning we retain scene consistency to reduce search errors, avoiding distribution shifts that might be introduced by cross-domain data.

Let N(z,Q)N(z,Q) denote the nearest neighbor of zz in support set QQ. The two prediction branches match the neighbor features from the opposite augmented-view support sets:

LIDNNi=12D(p1i,stopgrad(N(z,Qz2in)))+12D(p2i,stopgrad(N(z,Qz1in)))\mathcal{L}_{\text{IDNN}}^i=\frac12\mathcal{D}(p_1^i,\texttt{stopgrad}(N(z,Q_{z_2}^{\mathrm{in}})))+\frac12\mathcal{D}(p_2^i,\texttt{stopgrad}(N(z,Q_{z_1}^{\mathrm{in}})))

Here, the same SimSiam-style stop-gradient design is used to maintain symmetry and prevent collapse.

IDNN's aim is to bring neighboring samples that may share the live or spoof class closer, even when their identities differ. Without labels, though, a retrieved neighbor may belong to the wrong class.

Searching within one source dataset reduces interference from shifts between datasets, though materials and capture conditions may still vary within that domain.

It compensates for the cross-sample fragmentation that SRM has not resolved, while preserving the image details of the domain itself as training cues.

Cross-domain Nearest Neighbors

In-domain neighbor search aims to bring potentially similar live or spoof representations closer across identities within one domain.

However, in real-world applications, FAS systems are often deployed across various devices and scenarios, with input data exhibiting strong domain gaps, including differences in image resolution, camera equipment, lighting conditions, and background styles.

These cross-domain factors cause the learned feature distributions to be highly heterogeneous. If the raw embeddings are directly used for cross-dataset neighbor search, it could lead to semantic mismatches and even erroneously bring closer samples from different spoof types.

Therefore, the authors introduced the Cross-domain Nearest Neighbor (CDNN) module as the third positive sample path in contrastive learning.

To reduce distribution differences, CDNN standardizes global and local search features using each domain's statistics. This aligns means and variances, without guaranteeing a Gaussian distribution. The two types of features use their respective statistics:

z^=z−μz,dσz,d2+ϵ,v^=v−μv,dσv,d2+ϵ\hat{z} = \frac{z - \mu_{z,d}}{\sqrt{\sigma_{z,d}^2 + \epsilon}}, \quad \hat{v} = \frac{v - \mu_{v,d}}{\sqrt{\sigma_{v,d}^2 + \epsilon}}

μz,d,σz,d2\mu_{z,d},\sigma_{z,d}^2 and μv,d,σv,d2\mu_{v,d},\sigma_{v,d}^2 describe the domain’s global and local features separately; ϵ\epsilon stabilizes the denominator. The local search feature pools the merged patch-combination features.

After normalization, the similarity calculation between samples no longer relies solely on the global representation. Instead, the distance between z^\hat{z} and v^\hat{v} is combined:

dsearch=∥z^i−z^j∥2+∥v^i−v^j∥2d_{\mathrm{search}} = \| \hat{z}_i - \hat{z}_j \|_2 + \| \hat{v}_i - \hat{v}_j \|_2

Considering both global and local features is meant to reduce the misleading effects of identity and style at any one scale.

Rank candidates by the Euclidean search score dsearchd_{\mathrm{search}}, then retrieve neighbor features from the two augmented cross-domain support sets. The retrieved neighbor can still belong to the wrong live/spoof class. Training uses negative cosine similarity D\mathcal{D}, which is different from the search metric:

LCDNNi=12D(p1i,stopgrad(N([z,v],Qz2cr)))+12D(p2i,stopgrad(N([z,v],Qz1cr)))\mathcal{L}_{\text{CDNN}}^i=\frac12\mathcal{D}(p_1^i,\texttt{stopgrad}(N([z,v],Q_{z_2}^{\mathrm{cr}})))+\frac12\mathcal{D}(p_2^i,\texttt{stopgrad}(N([z,v],Q_{z_1}^{\mathrm{cr}})))

CDNN's aim is to reduce the bias between scenes, not merely add more training examples. An incorrectly matched neighbor can still introduce noise.

Training Objective Integration

The three loss terms described above are combined into the final objective function:

L=LSRM+λ1⋅LIDNN+λ2⋅LCDNN\mathcal{L} = \mathcal{L}_{\text{SRM}} + \lambda_1 \cdot \mathcal{L}_{\text{IDNN}} + \lambda_2 \cdot \mathcal{L}_{\text{CDNN}}

To avoid unreliable neighbors early in training, the two neighbor losses are delayed: λ1\lambda_1 stays at 0 for the first T1=30T_1=30 epochs and λ2\lambda_2 for the first T2=60T_2=60, then each switches to 1. This lets SRM establish an initial representation first.

UDG-FAS does not pursue a new backbone; it focuses on “filtering signals” and aligning representations during unlabeled pretraining. The neighbor searches still need source-domain information, and the downstream fine-tuning experiments still use source-domain labels.

Discussion

To comprehensively evaluate the generalization ability of the proposed method in different application scenarios, the UDG-FAS paper designed six experimental protocols.

Protocol-1: Cross-Domain Generalization with Low Labeled Data

This protocol assumes access to unlabeled images from several source domains, followed by fine-tuning with only some labeled source data. Specifically, the model pretrains on three of I/O/C/M, fine-tunes on 5%, 10%, 20%, 50%, or 100% of the labels from those same sources, and tests on the held-out target. The fractions are split by identity; the target is not used for fine-tuning. Another setting keeps all live labels while varying the labeled spoof fraction.

UDG-FAS beats the ImageNet initialization and listed unsupervised baselines on all four cross-domain tests, even when few source samples are labeled for fine-tuning. This is consistent with the aim of reducing identity and domain bias.

Moreover, when combined with existing DG methods (such as SSDG-R), UDG-FAS can further improve overall performance, demonstrating its excellent modular integration and generalization potential.

Protocol 2: Feature Quality without Fine-tuning

This protocol tests the quality of the pretrained features without fine-tuning the encoder. It uses a k=10k=10 kNN classifier on the target domain. The supplementary material specifies labeled reference examples for that evaluation, so it tests separability after unlabeled pretraining, not label-free prediction from start to finish.

UDG-FAS leads the listed unsupervised methods in all four kNN tests. On O&C&M → I, its AUC is 94.75%, also above the 92.69% SimSiam result fine-tuned with all labels in Table 1. The evaluation procedures differ, and downstream classification still uses class information.

The result supports useful FAS representations learned without class labels, while the kNN evaluation still relies on labels to identify the classes.

Protocol 3: Pre-training Performance Extension with Large-Scale Unlabeled Data

Here the authors ask what happens when much more unlabeled data are available. They add 100,000 or 200,000 CelebA-Spoof (CA) images to pretraining, or use the web-crawled real-face subset of CA. The model is then fine-tuned with all labels from the smaller source datasets and tested on the held-out domain.

Adding 100,000 and then 200,000 CA images lowers HTER in all four directions. The web-crawled real-face subset also lowers all four HTERs, though less than the 200,000-image addition, and the first AUC falls from 97.31% to 97.08%. The table does not directly measure resistance to noisy data.

Under this pretraining and subsequent full-label fine-tuning setup, extra unlabeled data help. The experiment does not directly compare data diversity with annotation quality.

Protocol 4: Generalization Ability Test with Limited Data Sources

With source data harder to collect, Protocol 4 uses only two datasets for unlabeled pretraining and labeled fine-tuning, then tests on the other unseen pair. It evaluates both O&M → C&I and the reverse direction.

UDG-FAS has the lowest HTER in both directions, and the highest AUC on O&M → C&I. In the reverse C&I → O&M direction, its 88.81% AUC is below DF-DM†'s 89.76%. The result supports this limited-source setting, not a claim that every metric or the feature space's stability is best.

Protocol 5: Dual Generalization Challenge across Data Sources and Attack Types

This protocol combines cross-domain and cross-attack type shifts. Training uses specified source datasets and only some known attack types for pretraining and fine-tuning; testing faces another domain with unseen spoof types.

Across the nine attack categories in Table 6, UDG-FAS has the highest overall mean AUC at 92.43% (standard deviation 6.86). Other methods still win on individual categories, including the CASIA-MFSD Video and Warped attacks.

This supports transfer to unseen 2D attack types in the paper's setting.

Protocol 6: Generalization Test for Unseen 3D Attacks and Large-Scale Data

This protocol asks whether the model can recognize 3D mask attacks it has never seen. It pretrains on unlabeled O/C/I/M, fine-tunes on their labels, and tests on 3DMAD (D) and HKBU-MARs (H). A separate setup uses O/C/M as sources and tests on CA, which includes unseen 3D attacks. These are changes in attack type, not a test of a new sensor modality.

On 3DMAD, UDG-FAS has the table's best 0.29% HTER and 99.95% AUC. On HKBU-MARs, its 27.36% HTER is just above SSAN's 26.98%, and its 80.31% AUC is below RFMetaFAS's 81.64%. The two mask datasets give very different results.

In the large-scale CA evaluation, UDG-FAS's 21.35% HTER and 87.26% AUC beat the listed comparison methods. Those are results under this protocol, not a direct measure of reliability in deployment.

Conclusion

UDG-FAS does not seek breakthroughs in model architecture or training processes but instead poses a more fundamental question:

What remains if we no longer rely on annotations?

The answer proposed by this paper returns to the representation itself.

It does not propose a new backbone, nor does it design complex loss terms. Instead, through a series of arrangements, it allows the model to gradually build a semantic space capable of navigating identity and style noise under unsupervised conditions.

The contribution of this work is not merely a set of good results but a new perspective:

When supervisory information is unavailable, the responsibility of feature design can no longer be shifted to classifiers or regularization strategies.

We must make choices and limitations at the beginning of learning, guiding the model in the right direction, rather than waiting for the data to tell it what is right.

In the longer run, methods like this may lessen the traditional FAS training pipeline's reliance on labeled attack samples. This study still uses unlabeled attack images during pretraining and labels for fine-tuning or kNN evaluation.

After all, the semantic boundary of whether something is live or spoofed should never just be the surface texture of a specific attack type, but a semantic resonance that spans identity, modality, and data sources.

And this, perhaps, is the true beginning of unsupervised learning.