Back to paper library
WACV 2023Face anti-spoofingPaper notes

DiVT

All-Star Championship

Original paper

Domain Invariant Vision Transformer Learning for Face Anti-Spoofing

Paper authors

Chen-Hao Liao · Wen-Cheng Chen · Hsuan-Tung Liu · Yi-Ren Yeh · Min-Chun Hu · Chu-Song Chen

Publication
WACV 2023Conference
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@inproceedings{faceantispoofingdivt2023,
  title = {{Domain Invariant Vision Transformer Learning for Face Anti-Spoofing}},
  author = {Chen-Hao Liao and Wen-Cheng Chen and Hsuan-Tung Liu and Yi-Ren Yeh and Min-Chun Hu and Chu-Song Chen},
  year = {2023},
  booktitle = {WACV},
  url = {https://openaccess.thecvf.com/content/WACV2023/html/Liao_Domain_Invariant_Vision_Transformer_Learning_for_Face_Anti-Spoofing_WACV_2023_paper.html}
}
On this page

It's rare to see a paper where the authors list several of Taiwan's top universities.

NTU, NCKU, NTHU, NKNU... Huh? And even E.SUN Bank.

Problem Definition

In the previous section, we discussed a lot about Domain Generalization (DG), and this paper serves as a bridge to extend the concept of DG from CNN architectures to Transformer models.

The new generation of architectures will carry on the legacy of the past and continue forward.

Looking back at existing DG-FAS solutions, they can generally be categorized into two directions.

The first approach attempts to bring "real faces" closer together.

Regardless of which domain they come from, real faces should share some intrinsic life-like features. In the feature space, these real samples from different domains are clustered into the same semantic group. The key idea here is to establish a stable anchor point across different scenes.

The second approach focuses on splitting "attack types."

Different attack methods have different disguise strategies, which lead to varying visual features. These features are pushed apart and separated during the learning process to prevent the model from confusing or misjudging them.

Thus, the entire feature space appears as follows:

  • Real faces are concentrated at the center, as the core of alignment.
  • Various spoofed faces are dispersed at the boundaries, separated from each other.

In practice, most of these methods are still built on CNN architectures. For example, SSDG uses single-sided adversarial learning to stabilize real face features, ANRL modifies normalization logic to combat domain shift, and SSAN separates style and content for cross-domain contrastive learning. There are also generative methods like FGHV, which use Gaussian distribution assumptions to approximate real faces.

These designs are sophisticated and effective but come with complex learning mechanisms and high resource costs.

All of these efforts are based on a shared assumption:

The model has never seen any data from the target domain.

While this assumption is stringent, it is closer to real-world applications. The real world doesn't wait for you to gather all the variables; it just throws an unseen exception at you, requiring your model to immediately provide an answer.

This is why some have started turning their attention to Transformers.

Compared with a local CNN operation, self-attention can connect distant image regions directly and model their relationships more flexibly. That gives Transformers a theoretical appeal in cross-domain learning, where visual patterns can vary over long distances. Generalization also depends on architecture, data, and training.

However, current Transformer attempts for FAS are still limited. Some directly apply ViT, others use it as a teacher model while the student model remains CNN-based. Most designs also don't make special adjustments for domain generalization.

So, if we want to design a truly cross-domain generalized Transformer, where should we start?

Solution

The core claim of this paper is actually very simple:

Use a lightweight yet sufficiently smart Transformer architecture to bring "real faces" together and separate "attacks."

Model Architecture

The overall approach is built on MobileViT, a hybrid architecture that combines CNN and Transformer.

MobileViT-S uses MobileNetV2-style inverted residual blocks for initial feature extraction and downsampling, alongside local convolution and global attention in MobileViT blocks. This lets it capture facial detail and broader structure within a comparatively small backbone. It does not simply attach a complete independent MobileNetV2 to a Transformer. Parameters and FLOPs describe model size and arithmetic cost; device latency still needs measurement.

The first step for cross-domain generalization is to achieve consensus between "real faces" from different scenes.

Assume we have datasets from KK domains: D1,D2,…,DKD_1, D_2, \ldots, D_K, where each dataset contains real faces and several types of attack samples.

The authors merge all the real faces into:

DR=⋃k=1KDkrealD_R = \bigcup_{k=1}^K D^\text{real}_k

Then, using the Transformer backbone, these samples are mapped to feature space, resulting in a set of embedding vectors:

ER=π(DR)E_R = \pi(D_R)

The authors' approach is to treat the origin [0]d[0]^d as the center of real faces, designing a simple ℓ1\ell_1-norm loss to pull these features towards the origin:

LDiC=1N∑i=1N1[xi∈DR]⋅∥π(xi)∥1\mathcal{L}_\text{DiC} = \frac{1}{N} \sum_{i=1}^N \mathbb{1}[x_i \in D_R] \cdot \|\pi(x_i)\|_1

The concentration idea resembles Center Loss, but fixes the genuine-face center at the origin instead of learning moving class centers. NN is the entire mini-batch size; only genuine samples contribute to this term. Attacks remain supervised by the multi-class objective below, which requires labels for the known training attack types.

Conceptually, the following diagram shows how real faces from different domains are concentrated at the center, rather than being scattered along a specific domain axis, while attack samples are ignored.

After bringing real faces closer, the next step is to push away the spoofed faces.

For each attack type, we combine data from different domains into a single class:

Dc=⋃k=1KDkc,c∈{1,…,C}D^c = \bigcup_{k=1}^K D^c_k, \quad c \in \{1, \ldots, C\}

Then, we add a classification head and perform C+1C+1 class classification (real face + various attacks).

The cross-entropy loss here is:

LDiA=−1N∑i=1N∑c=0Cy^iclog⁡yic\mathcal{L}_\text{DiA} = -\frac{1}{N} \sum_{i=1}^N \sum_{c=0}^{C} \hat{y}^c_i \log y^c_i

where y^ic\hat{y}^c_i is the one-hot label (c=0 represents real face), and yicy^c_i is the softmax output of the model for each class.

The task of this classifier is not just to "identify" the attack type but more importantly to further separate real and fake samples in the latent space. The benefit of this approach is that attack samples are dispersed, while maintaining distance from real faces, enhancing the decoupling ability across domains.

The final loss function is a weighted combination of the two:

Ltotal=LDiA+λLDiC\mathcal{L}_\text{total} = \mathcal{L}_\text{DiA} + \lambda \mathcal{L}_\text{DiC}

During inference, only the prediction probability for real faces from the classification head is used as the output, without requiring additional modules.

This approach does not introduce adversarial learning or rely on style separation or feature generation. Instead, it achieves stable and effective generalization with two intuitive objective functions and a lightweight architecture.

Discussion

The authors tested their method on four commonly used public FAS datasets: CASIA-FASD, MSU-MFSD, Replay-Attack, and OULU-NPU.

These datasets come from different capture devices, attack methods, and lighting conditions, each with distinct domain differences, making them a representative test set for evaluating generalization capabilities.

This study discards the validation splits of Replay-Attack and OULU-NPU, uses their training/test data, and uses all data from the other two datasets. In leave-one-domain-out evaluation, the final held-out domain is excluded from training. This is the study’s protocol, not a universal split rule for DG-FAS.

The table counts genuine and attack videos, not frames or subjects. Training samples frames randomly from videos and balances real/spoof examples from the source datasets at each step.

The evaluation metrics used are two common ones:

  • HTER (Half Total Error Rate): The mean of FAR and FRR; lower is better. Inspect FAR and FRR separately at the chosen decision threshold.
  • AUC (Area Under the Curve): Area under the ROC curve, measuring score-ranking discrimination. It does not directly measure training-run stability or guarantee performance at low false acceptance rates.

Implementation Details

All video frames were aligned and cropped using MTCNN, then resized to 256×256256 \times 256. During training, the authors randomly sample one frame per video and balance the numbers of real and spoof examples.

The MobileViT-S version was chosen as the backbone, pre-trained on ImageNet-1K. The optimizer used was Adam, with a learning rate of 10−410^{-4} and weight decay set to 10−610^{-6}. The balance coefficient λ between the two loss functions was set to 0.2.

Leave-One-Out Test Results

The table above shows the performance under the leave-one-out setting. In this setup, three datasets from the four are used for training, and the remaining one is used as the test set, simulating the situation where the model encounters a new domain.

Most methods are frame-based, and this paper's method follows suit; the only exception is NAS-FAS, which is based on video signals and theoretically benefits from additional temporal cues.

Among the contemporary methods in Table 2, DiVT has the lowest HTER in all four settings, with absolute gains exceeding 3 percentage points in two settings.

The only time DiVT did not win was in the AUC metric for I&C&M to O, where NAS-FAS slightly outperformed it. However, this method is video-based, making it a different level of comparison when compared to pure image-based methods.

Small Sample Setting

The reduced-data experiment uses only MSU-MFSD and Replay-Attack as sources and tests on each remaining domain. Small sample here means fewer source domains and less training data, not a few-shot setting with only a handful of examples per class.

This setup is closer to the data challenges encountered during actual deployment.

The authors retrain SSDG-R and SSAN-R. On M&I → C, DiVT has 20.11% HTER versus SSDG-R’s better 19.86%, but slightly higher AUC, 86.71% versus 86.46%. On M&I → O, its 23.61% HTER and 85.73% AUC are both better. These error rates also show why beating a baseline is not sufficient for deployment.

Ablation Study

The following analysis focuses on three questions:

  1. How significant are the performance differences between different Transformer architectures?
  2. How much do the two loss functions contribute to the performance?
  3. Would changing the classification logic yield better or worse results?

Comparison of Different Architectures

The results are shown in the table above. The upper part shows the performance of various architectures using only binary cross-entropy loss, while the lower part shows the results after incorporating the proposed losses.

Key differences observed include:

  • DiVT-V, DiVT-S, and DiVT-M use ViT-Base, Swin-T, and MobileViT-S, respectively. With the paper's losses, their mean HTERs are 12.11%, 9.12%, and 7.07%, below DiVT-ResNet's 17.25%. This supports the tested configurations without isolating the causal contribution of attention.
  • DiVT-M has the best mean HTER/AUC, 7.07%/97.34%, not every column: DiVT-S is better on O&M&I → C. The later resource table also gives DiVT-M the fewest parameters. Convergence time requires a separate measurement.
  • ViT-Base is weaker on average among the tested Transformer configurations. Less local inductive bias and a large model relative to the data are proposed explanations.
  • ViT-Tiny lowers mean HTER from ViT-Base's 12.11% to 11.47%, consistent with model scale being one possible factor.

The success of MobileViT also reinforces a key point: Convolutions are not useless; they should just appear in the right places.

Effect of Loss Function Combinations

The authors then tested the combinations of the two core loss functions in the proposed method (DiC loss and DiA loss), comparing the results when used separately, not used, and used together.

Observations include:

  • Replacing binary classification with DiA raises mean AUC from about 92.27% to 95.99%, an increase of roughly 3.72 percentage points.
  • Adding DiC on top of DiA raises mean AUC to about 97.35%, another 1.36 percentage points. Adding DiC to the binary baseline instead gives about 94.54%; these gains use different baselines.
  • Both together give the best average, with individual exceptions: DiA alone has 2.62% HTER on O&C&I → M versus 2.86% jointly, and DiC with binary classification has 94.11% AUC on I&C&M → O versus 94.04% jointly.

Would Changing the Classification Logic Be Better?

Keeping DiC loss, the authors also explored whether the design of the classification task itself affected the results. Besides binary classification (real vs. fake), they also tested:

  • Classification by domain (real + spoof from each domain)
  • Classification by attack type (real + various attack types)
  • Combined classification by domain and attack type

Attack-type classification has the best average, but domain classification is better on held-out OULU: 10.10% HTER / 96.43% AUC versus 13.06% / 94.04%. This is a tradeoff between average and individual-domain performance.

This supports DiVT's "unified real face, split spoof faces" classification design in these experiments, though the advantage varies by target domain.

What If We Added Adversarial Loss?

Some might ask why not add adversarial loss, like SSDG or SSAN?

Domain-adversarial training worsens three HTER settings but improves O&M&I → C: HTER 8.67% → 7.33% and AUC 96.92% → 97.56%. The paper reports no significance test. Its average benefit is limited; the following are the authors’ possible explanations, not verified causes:

  • DiVT’s feature space is already stable enough without additional competition.
  • The best practices for using Transformers in adversarial learning have not yet been fully developed.

Comparison of Computational Resources

Table 8 reports 4.94M parameters and 2.00G FLOPs for DiVT-M, with mean HTER/AUC of 7.07%/97.34%. This is a favorable tradeoff among the listed models. FLOPs do not equal latency, energy use, or peak memory; the paper does not validate performance on every intended endpoint device.

Although ViT-Tiny has fewer FLOPs (1.26G), it has more parameters (5.52M) and did not perform as well on average as MobileViT in these four tests. This shows that the architecture best suited for DG-FAS is not just lightweight, but also able to capture spatial patterns effectively.

Conclusion

Methods targeting domain generalization are no longer just about "model performance" but about "methodology choices."

This paper avoids the complex chess game of adversarial training and does not rely on synthetic data to enhance visual cues. Instead, it opts for a near-geometric abstraction, pulling space straight and clearly defining features in the coordinate system.

Beyond the technical level, this design approach reveals a sensitivity to real-world deployment:

It’s not about stacking modalities or using heavy architectures. It’s about finding the critical point where just enough is done to maintain the rhythm.

Future challenges will resemble combinatorial boundary problems, with the intersection of domain and attack dimensions becoming increasingly complex. This kind of minimalist and clear representation strategy could be the starting point for reorganizing the problem space.