Disentangled Representation with Dual-stage Feature Learning for Face Anti-spoofing
- Publication
- WACV 2022Conference
- First submitted to arXiv
- arXiv:2110.09157
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{faceantispoofingdualstage2021,
title = {{Disentangled Representation with Dual-stage Feature Learning for Face Anti-spoofing}},
author = {Yu-Chun Wang and Chien-Yi Wang and Shang-Hong Lai},
year = {2021},
eprint = {2110.09157},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2110.09157}
}
Disentangled representation learning has become an increasingly popular topic.
The concept of disentanglement itself is relatively agreed upon: it aims to extract spoofing features.
But what about everything else besides the spoof features?
Problem Definition
In most face anti-spoofing systems, the model tries to infer one thing from an image: is this a live person or a spoof?
However, this decision is actually inferred from a tangled mixture of signals: the face material, residual lighting, or compression artifacts.
In other words, almost all information that can sneak in, does sneak in.
This is exactly the core issue disentangled representation learning seeks to address:
Not only to answer true or false, but to clarify what exactly you have seen.
In recent years, many studies have proposed various disentanglement strategies, and we have reviewed several related papers.
Some split the latent space into two parts, separating liveness features from content features; others try to decompose spoof traces into multi-scale residuals simulating spoof artifacts. But nearly all methods rely on one assumption:
We can complete both feature separation and reconstruction in a single stage.
The authors question that assumption: separation and reconstruction do not always happen so neatly when learned together.
- Once disentanglement and classification objectives overlap too much, models tend to take shortcuts.
- Adversarial training can encourage latent space separation but often induces gradient oscillations, making learning highly unstable.
- Auxiliary cues such as depth maps and rPPG signals need evaluation across attack types.
In other words, we force the model to distinguish spoofing without providing a stable way for it to learn the spoofing syntax.
It can see real or fake, but cannot understand the syntactic differences between them; it can classify most samples correctly but cannot predict how the next attack might morph. The concept of disentangled representation in practice still suffers from architectural instability, signal dependency, and unpredictability for unknown attacks.
The authors of this paper propose:
Why not learn in stages?
Slowly but surely is faster.
Solution
When facing unknown attacks, simple binary classification is far from enough.
The authors train in two stages: first learn a reconstruction feature from genuine faces, then extract spoof-related features from real and attack faces. The figure above shows the process; let us go through it step by step.
Learn Liveness Information First
The authors first build a Live-info Network for autoencoding learning.
- Input: only live face samples
- Encoder : extracts liveness features
- Decoder : tries to reconstruct input, outputting
In this first stage, contains only genuine faces, and the network minimizes reconstruction error:
The network sees no attack faces at this stage, so learns features for reconstructing the genuine faces in training.
This encoder will later be used as a fixed-weight liveness feature extractor, introduced into the next stage.
Dual-Stream Parallel Reconstruction
In the second stage, spoof samples are introduced into training, forming a dual-stream parallel reconstruction disentanglement module:
-
Live-encoder
- Loads the weights trained in the first stage to extract liveness features .
- These weights are frozen to preserve the representation learned in stage one.
-
Spoof-encoder
- Dedicated to extracting spoof features , i.e., attack signals absent or subtle in real faces.
-
Reconstructor
- Attempts to reconstruct the input image based on the fused features and .
The authors use reconstruction loss and adversarial loss so can reconstruct the input and can focus on spoof differences:
-
Reconstruction Loss
-
Generator Loss
-
Discriminator Loss
Here, is the discriminator, distinct from the reconstruction decoder ; it distinguishes real images from generated ones.
This adversarial step encourages more realistic reconstructions. Alongside the other losses, it aims to separate live features from spoof features .
Extracting Spoof Clues
The authors connect to a U-Net-like decoder , which attempts to convert spoof features into a spoof map. To maximize the spoof map’s ability to distinguish "live vs. spoof," they introduce a triple loss design:
-
Triplet Loss
The goal is to pull together intra-live samples while pushing apart live and spoof samples.
These losses operate on the last three layers of . Genuine samples serve as anchors , with genuine positives and attack negatives . The normal and hard triplet losses, and , are added.
-
Minimizing Residual Spoof Loss ()
For live faces, no significant spoof patterns should be generated, so the spoof map is encouraged to approach zero:
-
Auxiliary Classifier Loss ()
The spoof map is overlaid on the input image and passed to auxiliary classifier ; the paper describes an overlay, not channel concatenation.
The binary cross-entropy above corrects the sign of the second term, also misprinted in the paper’s Eq. (9). The overall loss is:
The paper sets the weights as .
Training and Inference
-
Training procedure
- Stage one: train and only with live face samples.
- Stage two: freeze , then train the added , synthesis decoder , discriminator , spoof-map decoder , and auxiliary classifier through reconstruction, adversarial, and spoof-map supervision.
-
Inference
- Use only and to generate the spoof map.
- The average value of the spoof map is taken as the spoof score.
- Samples with scores above a threshold are classified as spoof attacks.
On a GeForce GTX 1080, the authors report an inference speed of 121.48 ± 1.4 FPS.
This design avoids labels for auxiliary tasks such as depth or rPPG, but still uses genuine/attack labels. Generalization to unknown attacks must be measured.
Discussion
Cross-type Experimental Performance
SiW-M uses leave-one-attack-type-out evaluation: 12 attack types and 80% of genuine videos are used for training, with the remaining attack type and 20% of genuine videos for testing. Mean ACER is 9.65 ± 7.15%, and EER 8.63 ± 5.18%, both below the listed comparison methods. Funnyeye and Paperglasses remain difficult, with ACER of 24.37% and 20.64%.
The MSU-MFSD, CASIA-MFSD, and Replay-Attack experiments also leave out one attack type. The method reaches the table's highest mean AUC, 98.0 ± 4.26% across nine tests. This is cross-type evaluation, rather than evaluation on an entirely unseen dataset. MSU print attacks reach 86.78% AUC, above the listed methods but the weakest of this method's nine results.
The average supports the dual-stage approach.
Same-type Experimental Performance
SiW’s three protocols cover pose/expression, presentation media, and attack types. The original section title is retained, but Protocol 3 is a cross-type test:
- Protocols 1 and 2: Both achieve 0% ACER. Protocol 1 ties DST, while Protocol 2 ties DST and LGC.
- Protocol 3: ACER is 3.58 ± 3.93%, worse than LGC’s 1.19 ± 1.39% and HMP’s 2.45 ± 0.68%. The authors suggest that training on only one attack type may constrain disentanglement; this is a proposed explanation, not an isolated causal result.
Results on Protocols 1 and 2 should be assessed separately from transfer with a single training attack type.
Visualization of Disentangled Features
To illustrate the disentangled features, the authors performed a “face swap” experiment.
They preserve each image’s live feature and exchange the two images’ spoof features, producing and . They do not exchange a live feature with a spoof feature.
The results show that after swapping, the real face acquires spoof textures such as “stripes or grayscale imbalance,” while the original spoof face partially recovers natural skin tone.
The examples support the interpretation that spoof features carry some attack texture.
Spoof Map Analysis
Finally, to observe what the disentangled spoof feature looks like, the authors decoded it via a U-Net style decoder to generate detection maps.
In the figure's genuine-face examples, the spoof maps are nearly zero. In its print-attack examples, bright regions appear around the forehead, eyes, nose, and mouth.
These maps provide visible image-space cues; whether each highlighted area corresponds to a physical attack mechanism requires separate validation.
Ablation Study
Because DualStage employs many loss functions, the authors conducted ablation experiments on their effects and compared results with other methods.
Removing the first stage or the adversarial loss reduces mean AUC across the nine tests from 98.0% to 95.01% or 94.0%, respectively. Removing all triplet loss gives 95.01%; retaining only normal triplet loss gives 97.25%.
These ablations support contributions from genuine-face pretraining, adversarial loss, and both triplet losses in the tested setting. Removing the auxiliary classifier also brings mean AUC down to 93.3%.
Conclusion
DualStage proposes a straightforward design:
Split the disentanglement process into two stages — first stably learn liveness features, then gradually peel off spoof signals.
It needs no extra depth maps or rPPG supervision. From the images themselves, it produces spoof maps useful for classification and shows strong results on several public cross-type tests. That gives the method a place even without support from multiple sensing modalities.
Of course, it still has drawbacks: performance is less satisfactory in scenarios with extremely sparse attack types (e.g., SiW Protocol 3). Future improvements may require further handling of seemingly “irrelevant” but actually disruptive feature fragments.
As a methodological proposal, it leaves an important message:
Disentanglement does not have to be done all at once.
The model can learn gradually, as long as it is given the right sequence.