Back to paper library
arXiv 2025AAAI 2025Face anti-spoofingPaper notes

I-FAS

The Final Chapter of Classification

Original paper

Interpretable Face Anti-Spoofing: Enhancing Generalization with Multimodal Large Language Models

Paper authors

Guosheng Zhang · Keyao Wang · Haixiao Yue …

Show all 8 authors

Guosheng Zhang · Keyao Wang · Haixiao Yue · Ajian Liu · Gang Zhang · Kun Yao · Errui Ding · Jingdong Wang

Publication
AAAI 2025Conference
First submitted to arXiv
arXiv:2501.01720
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@misc{faceantispoofingifas2025,
  title = {{Interpretable Face Anti-Spoofing: Enhancing Generalization with Multimodal Large Language Models}},
  author = {Guosheng Zhang and Keyao Wang and Haixiao Yue and Ajian Liu and Gang Zhang and Kun Yao and Errui Ding and Jingdong Wang},
  year = {2025},
  eprint = {2501.01720},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.01720}
}
On this page

Face anti-spoofing is commonly treated as a binary classification task: Real or Spoof.

However, if we ask "Why?", most models cannot provide an answer.

In fact, they do not even allow you to ask "Why?".

Problem Definition

Existing face anti-spoofing models are mostly trained on a single objective — classifying input images as either real or fake.

This logic has been used for years, but frequently fails in scenarios such as:

  • Large style gaps between test scenes and training data
  • New spoofing techniques (e.g., high-definition replay, 3D masks)
  • User groups with cultural, device, or geographic differences

Researchers have tried various approaches to strengthen generalization ability, including:

  1. Domain Adaptation (DA): Exposing the model to target domain data for feature alignment
  2. Domain Generalization (DG): Learning stable feature distributions from multiple source domains
  3. Incremental Learning (IL): Introducing new data domains while preserving prior knowledge

Yet, regardless of the method, the essence remains extracting "liveness features" from images — features that the models themselves mostly cannot interpret.

This is the core issue:

We train a system that can judge but cannot explain.

It can tell you "this is fake," but cannot specify "where it is fake" or "why it is fake."

We let the model see the image but never taught it how to tell a reasonable story.

Ironically, human recognition is precisely based on storytelling.

Solution

"Is this real?"

For face anti-spoofing models, this question should not only have an answer but also an explanation.

The I-FAS framework is built on this belief, translating the FAS task into an "image question answering" problem.

To make the model answer like a human, three aspects need to be addressed: language construction, feature alignment, and training strategy.

Spoof-aware Captioning and Filtering (SCF)

Most general captioning models only focus on facial expressions, clothing, or background, but ignore key spoofing clues.

As illustrated below:

The traditional captioner output is denoted as TFT_F, while after fine-tuning by the authors, the spoof-aware captioner is TST_S.

As shown, traditional captions tend to be vague, whereas the fine-tuned spoof-aware captioner mentions an attack medium such as screen or paper in these examples.

The SCF process is as follows:

  • Aggregate 12 public datasets into DD, labeled with real or fake samples (Ii,Yi)(I_i, Y_i).
  • Build a dictionary of spoof keywords KK, categorized by attack types.
  • Use a general captioner CGC_G to generate initial descriptions TFT_F for all fake images.
  • Filter out captions that do not contain corresponding keywords, keeping a subset DSD_S that clearly reflects attack features.
  • Fine-tune CGC_G on DSD_S to obtain the spoof-aware captioner CSC_S.
  • Finally, generate captions for all real images to form training data with semantic differences.

This strategy equips each data sample not only with a label but also with a semantic description for training. Whether that description explains the model's actual decision requires separate validation.

Multimodal Instruction Tuning with GAC

After completing the preparation of training data, we move on to the model training stage.

The overall model architecture is shown above. I-FAS consists of three main components:

  • Visual Encoder EVE_V: Extracts features XV∈RN×DX_V \in \mathbb{R}^{N \times D} from input image I∈RH×W×3I \in \mathbb{R}^{H \times W \times 3}.
  • Cross-modal Connector PV→TP_{V \to T}: Maps XVX_V into the token space of the language model.
  • Language Model ΦT\Phi_T: Automatically answers questions and generates natural language descriptions based on visual features and textual prompts.

The visual encoder and language model in the diagram remain frozen; the GAC connects visual features to the language model.

Globally Aware Connector (GAC)

To achieve more comprehensive and detailed visual semantics, the authors propose the GAC module, which aims to transmit multi-level global features to the language model. The detailed process is as follows:

  • Extract gig_i (i.e., cls tokens) from each layer of the visual encoder, forming GV={g1,...,gL}G_V = \{g_1, ..., g_L\}.

  • Linearly project these into QVQ_V and concatenate with learnable query vectors QPQ_P:

    Q=Concat(QP,QV)Q = \text{Concat}(Q_P, Q_V)
  • Then, QQ undergoes multi-head self-attention (MSA) and multi-head cross-attention (MCA) combined with regional features XVX_V:

    Q′=Q+MSA(LN(Q))Q′′=Q′+MCA(LN(Q′),LN(XV))XT=Q′′+MLP(LN(Q′′))\begin{aligned} Q' &= Q + \text{MSA}(\text{LN}(Q)) \\ Q'' &= Q' + \text{MCA}(\text{LN}(Q'), \text{LN}(X_V)) \\ X_T &= Q'' + \text{MLP}(\text{LN}(Q'')) \end{aligned}

This means using QQ as the main query to attend to image features and find liveness-specific representations.

Typically, shallow features help capture texture details (e.g., moiré patterns, blur), while deep features tend to abstract semantics; GAC integrates these perspectives into the language output. Thus, the input to the language model is not a single feature, but a cross-modal representation that fuses visual depth, spatial, and semantic hierarchies.

Lopsided Language Model Loss (L-LM Loss)

Once the model learns to "speak," the next question is:

Should it speak the "result" first or the "reason" first?

The authors reformulate the FAS task into a single-turn question-answer format, with training data composed of three parts: (I,TQ,TA)(I, T_Q, T_A)

where:

  • TQT_Q is a fixed instruction sentence: "Is this photo of a real person?"

  • TAT_A is a two-segment answer format:

    TA=[TJudgment, TInterpretation]T_A = [T_{\text{Judgment}},\ T_{\text{Interpretation}}]

Here, TJudgmentT_{\text{Judgment}} is the model’s real/fake decision (“Yes” or “No”), and TInterpretationT_{\text{Interpretation}} is a semantic explanation generated by the captioner (CGC_G or CSC_S) to justify the judgment.

Using traditional LM loss, the model treats the entire TAT_A as continuous text for autoregressive learning.

This leads to two issues:

  1. Unclear semantic focus: The model may overly mimic the style of the caption at the sentence end, neglecting key judgment tokens.
  2. Noise hampers convergence: Explanations often contain detailed noise, offering limited help to the core FAS decision and slowing the model’s convergence on the main task.

To give the correct judgment more weight during training while still learning an explanation, the authors adopt an asymmetrically weighted language model loss called Lopsided LM Loss (L-LM Loss):

Ltotal=αLJudgment+(1−α)LInterpretationL_{\text{total}} = \alpha L_{\text{Judgment}} + (1 - \alpha) L_{\text{Interpretation}}

where:

  • LJudgmentL_{\text{Judgment}} is the token-level loss for the judgment part (Yes/No)
  • LInterpretationL_{\text{Interpretation}} is the loss for the explanation part
  • α∈[0,1]\alpha \in [0,1] is a hyperparameter controlling the training focus

Changing α\alpha adjusts the relative weight of judgment and interpretation tokens in one training objective. The paper does not describe a staged curriculum that first trains judgment and then gradually adds explanations. At inference, the probability of answering "Yes" is used as the live-face score.

The reported HTER/AUC and training-loss curves show better results and faster, less variable loss reduction with L-LM Loss in the tested setting.

Discussion

Cross-Domain Generalization Evaluation

First, Protocol 1 uses four classic datasets in a leave-one-domain-out test.

As shown in the table, I-FAS has the lowest HTER in all four tests, averaging 1.33%. Its AUC is highest in three of the four: for O&C&M→I, its 98.48% trails DiVT-M's 99.29%. These results support cross-domain performance under the tested protocol; the table alone cannot attribute the whole difference to natural-language supervision.

Protocol 2 pushes the experiment to an extreme: training only on CelebA-Spoof as a single source, while testing on 11 unseen datasets differing in attack types and sensing devices.

Experimental results:

Under this setting, I-FAS averages 93.71% AUC, 5.81 percentage points above FLIP's 87.90% and 10.89 points above ViTAF's 82.82%. For SURF-3DMask and SIW-M-V2, both HTER and AUC beat the listed comparisons. On HKBU-MARs-V1+, AUC edges out FLIP, but HTER is 18.64% versus FLIP's lower 17.25%.

Module Contributions

To verify the contribution of each module, the authors performed separate ablations, removing one of the three core components from the full model at a time. Results are shown above:

  • Removing SCF also reverts the target answer to Yes/No only. HTER rises by 3.17 percentage points and AUC falls by 3.30 points; this ablation changes both the caption supervision and the explanation output.
  • Removing GAC, while keeping the token count with learnable queries, raises HTER by 3.54 points and lowers AUC by 2.93 points. This supports the contribution of multilevel global features in this setting.
  • Replacing L-LM Loss with standard LM Loss raises HTER from 11.30% to 12.06% and lowers AUC from 93.71% to 92.81%. The loss curve is also more variable, though these results do not directly measure overfitting.

Caption Type Analysis

Does text diversity necessarily imply better generalization?

The authors conducted relevant experiments. The table compares various textual supervision formats, illustrating that semantic content in captions is key:

  • In I-FAS, judgment tokens alone yield 14.47% HTER and 90.41% AUC. Adding generic captions yields 14.23% and 90.22%, so the two metrics do not improve together. For FLIP, replacing templates with spoof-aware descriptions changes 18.73%/87.90% to 17.30%/89.15%.
  • With spoof-aware captions that mention attack clues, the listed mean AUC rises to 93.71%, compared with 90.22% for the corresponding generic-caption setting.
  • Replacing spoof-aware captions with generic captions (TFT_F) decreases performance, indicating that specificity of information matters more than corpus diversity.

L-LM Loss Hyperparameter Study

Among the tested α\alpha values, about 0.4 corresponds to the highest mean AUC and lowest mean HTER. The right-hand plot separately compares training loss with and without L-LM Loss:

  • If α\alpha is too high, interpretation has less weight in the objective, and the plotted cross-domain metrics worsen.
  • If α\alpha is too low, interpretation has more weight; the authors suggest caption noise may hinder learning, and the plotted α=0.2\alpha=0.2 is also worse than 0.4.

This comparison supports balancing the two losses. The curves alone do not show that the model has learned a structured reasoning process.

Explanation Capability Visualization

Earlier we compared spoof-aware captions with generic captions; the former mention media such as paper, tablet, or screen in the selected examples, providing more task-focused text supervision.

Finally, the figure below shows selected answers on SIW-M-V2 and Rose-Youtu after training on the single CelebA-Spoof source. They contain judgment and description, illustrating the output format. Whether the model actually used the stated clues for its judgment requires validation beyond these examples.

Conclusion

When models can only learn to answer "yes" or "no," we can only expect a confidence score output, but remain blind to why and based on which clues the decision was made.

This classification mindset, while helpful for system development and evaluation, traps models in a black-box framework, struggling to cope with cross-domain generalization, evolving attack types, and changing deployment scenarios.

This study attempts to redefine the output format of this task by introducing natural language question answering and caption-assisted supervision, enabling models not only to "answer true or false" but also to "explain their reasoning."

This shift in supervision syntax reflects a broader trend in computer vision:

From single-label to structured semantic descriptions.

Experimental results show that this semantic guidance brings substantial benefits. In Protocol 1's multi-source test, I-FAS has the lowest HTER in all four folds; in Protocol 2's single-source test, it leads the reported averages, with some per-dataset exceptions.

This suggests that language can aid learning in these tested settings, though the faithfulness of the generated explanations still needs verification. This rewrite driven by language is not merely the process of models learning to speak, but an opportunity to rethink the boundaries of the FAS task.

The moment the model begins to speak, perhaps not only FAS but

All classification tasks will eventually face a different destiny.