Back to paper library
arXiv 2021WACV 2022Face anti-spoofingPaper notes

DualStage

The Technique of Dual Disentanglement

Original paper

Disentangled Representation with Dual-stage Feature Learning for Face Anti-spoofing

Paper authors

Yu-Chun Wang · Chien-Yi Wang · Shang-Hong Lai

Publication
WACV 2022Conference
First submitted to arXiv
arXiv:2110.09157
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@misc{faceantispoofingdualstage2021,
  title = {{Disentangled Representation with Dual-stage Feature Learning for Face Anti-spoofing}},
  author = {Yu-Chun Wang and Chien-Yi Wang and Shang-Hong Lai},
  year = {2021},
  eprint = {2110.09157},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.09157}
}
On this page

Disentangled representation learning has become an increasingly popular topic.

The concept of disentanglement itself is relatively agreed upon: it aims to extract spoofing features.

But what about everything else besides the spoof features?

Problem Definition

In most face anti-spoofing systems, the model tries to infer one thing from an image: is this a live person or a spoof?

However, this decision is actually inferred from a tangled mixture of signals: the face material, residual lighting, or compression artifacts.

In other words, almost all information that can sneak in, does sneak in.

This is exactly the core issue disentangled representation learning seeks to address:

Not only to answer true or false, but to clarify what exactly you have seen.

In recent years, many studies have proposed various disentanglement strategies, and we have reviewed several related papers.

Some split the latent space into two parts, separating liveness features from content features; others try to decompose spoof traces into multi-scale residuals simulating spoof artifacts. But nearly all methods rely on one assumption:

We can complete both feature separation and reconstruction in a single stage.

The authors question that assumption: separation and reconstruction do not always happen so neatly when learned together.

  • Once disentanglement and classification objectives overlap too much, models tend to take shortcuts.
  • Adversarial training can encourage latent space separation but often induces gradient oscillations, making learning highly unstable.
  • Auxiliary cues such as depth maps and rPPG signals need evaluation across attack types.

In other words, we force the model to distinguish spoofing without providing a stable way for it to learn the spoofing syntax.

It can see real or fake, but cannot understand the syntactic differences between them; it can classify most samples correctly but cannot predict how the next attack might morph. The concept of disentangled representation in practice still suffers from architectural instability, signal dependency, and unpredictability for unknown attacks.

The authors of this paper propose:

Why not learn in stages?

Slowly but surely is faster.

Solution

When facing unknown attacks, simple binary classification is far from enough.

The authors train in two stages: first learn a reconstruction feature from genuine faces, then extract spoof-related features from real and attack faces. The figure above shows the process; let us go through it step by step.

Learn Liveness Information First

The authors first build a Live-info Network for autoencoding learning.

  • Input: only live face samples x∗∈PXx^* \in P_X
  • Encoder ELE_L: extracts liveness features FLF_L
  • Decoder DLD_L: tries to reconstruct input, outputting x^syn∗\hat{x}^*_{syn}

In this first stage, PXP_X contains only genuine faces, and the network minimizes reconstruction error:

LLive=Ex∗∼PX[∥x∗−x^syn∗∥22]\mathcal{L}_{Live} = \mathbb{E}_{x^* \sim P_X} \Big[\big\|x^* - \hat{x}^*_{syn}\big\|_2^2\Big]

The network sees no attack faces at this stage, so ELE_L learns features for reconstructing the genuine faces in training.

This encoder will later be used as a fixed-weight liveness feature extractor, introduced into the next stage.

Dual-Stream Parallel Reconstruction

In the second stage, spoof samples are introduced into training, forming a dual-stream parallel reconstruction disentanglement module:

  1. Live-encoder ELE_L

    • Loads the weights trained in the first stage to extract liveness features FLF_L.
    • These weights are frozen to preserve the representation learned in stage one.
  2. Spoof-encoder ESE_S

    • Dedicated to extracting spoof features FSF_S, i.e., attack signals absent or subtle in real faces.
  3. Reconstructor DsynD_{syn}

    • Attempts to reconstruct the input image x^syn\hat{x}_{syn} based on the fused features FLF_L and FSF_S.

The authors use reconstruction loss and adversarial loss so DsynD_{syn} can reconstruct the input and ESE_S can focus on spoof differences:

  • Reconstruction Loss

    Lrecon=Ex∼PX[∥x−x^syn∥22]\mathcal{L}_{recon} = \mathbb{E}_{x \sim P_X} \Big[\big\|x - \hat{x}_{syn}\big\|_2^2\Big]
  • Generator Loss

    LGen=Ex∼PX[(D(x^syn)−1)2]\mathcal{L}_{Gen} = \mathbb{E}_{x \sim P_X} \Big[(D(\hat{x}_{syn}) - 1)^2\Big]
  • Discriminator Loss

    LDis=Ex∼PX[(D(x)−1)2+(D(x^syn))2]\mathcal{L}_{Dis} = \mathbb{E}_{x \sim P_X} \Big[(D(x) - 1)^2 + (D(\hat{x}_{syn}))^2\Big]

Here, D(⋅)D(\cdot) is the discriminator, distinct from the reconstruction decoder DsynD_{syn}; it distinguishes real images from generated ones.

This adversarial step encourages more realistic reconstructions. Alongside the other losses, it aims to separate live features FLF_L from spoof features FSF_S.

Extracting Spoof Clues

The authors connect FSF_S to a U-Net-like decoder DmapD_{map}, which attempts to convert spoof features into a spoof map. To maximize the spoof map’s ability to distinguish "live vs. spoof," they introduce a triple loss design:

  1. Triplet Loss

    The goal is to pull together intra-live samples while pushing apart live and spoof samples.

    These losses operate on the last three layers of DmapD_{map}. Genuine samples serve as anchors fiaf_i^a, with genuine positives fipf_i^p and attack negatives finf_i^n. The normal and hard triplet losses, Ltnormal\mathcal{L}_t^{normal} and Lthard\mathcal{L}_t^{hard}, are added.

  2. Minimizing Residual Spoof Loss (Lr\mathcal{L}_r)

    For live faces, no significant spoof patterns should be generated, so the spoof map is encouraged to approach zero:

    Lr=Ex∼Plive[∥y^map∥1]\mathcal{L}_r = \mathbb{E}_{x \sim P_{live}} \Big[\big\|\hat{y}_{map}\big\|_1\Big]
  3. Auxiliary Classifier Loss (Lc\mathcal{L}_c)

    The spoof map y^map\hat{y}_{map} is overlaid on the input image xx and passed to auxiliary classifier CauxC_{aux}; the paper describes an overlay, not channel concatenation.

    Lc=E(x,y)[−y⋅log⁡(y^c)−(1−y)⋅log⁡(1−y^c)]\mathcal{L}_c = \mathbb{E}_{(x,y)} \Big[ -y \cdot \log(\hat{y}_c) - (1-y)\cdot \log(1-\hat{y}_c) \Big]

The binary cross-entropy above corrects the sign of the second term, also misprinted in the paper’s Eq. (9). The overall loss is:

L=λ1Lrecon+λ2LGen+λ3Lt+λ4Lr+λ5Lc\mathcal{L} = \lambda_1 \mathcal{L}_{recon} + \lambda_2 \mathcal{L}_{Gen} + \lambda_3 \mathcal{L}_t + \lambda_4 \mathcal{L}_r + \lambda_5 \mathcal{L}_c

The paper sets the weights as λ1=4, λ2=1, λ3=3, λ4=5, λ5=5\lambda_1=4, \ \lambda_2=1, \ \lambda_3=3, \ \lambda_4=5, \ \lambda_5=5.

Training and Inference

  1. Training procedure

    • Stage one: train ELE_L and DLD_L only with live face samples.
    • Stage two: freeze ELE_L, then train the added ESE_S, synthesis decoder DsynD_{syn}, discriminator DD, spoof-map decoder DmapD_{map}, and auxiliary classifier CauxC_{aux} through reconstruction, adversarial, and spoof-map supervision.
  2. Inference

    • Use only ESE_S and DmapD_{map} to generate the spoof map.
    • The average value of the spoof map is taken as the spoof score.
    • Samples with scores above a threshold are classified as spoof attacks.

On a GeForce GTX 1080, the authors report an inference speed of 121.48 ± 1.4 FPS.

This design avoids labels for auxiliary tasks such as depth or rPPG, but still uses genuine/attack labels. Generalization to unknown attacks must be measured.

Discussion

Cross-type Experimental Performance

SiW-M uses leave-one-attack-type-out evaluation: 12 attack types and 80% of genuine videos are used for training, with the remaining attack type and 20% of genuine videos for testing. Mean ACER is 9.65 ± 7.15%, and EER 8.63 ± 5.18%, both below the listed comparison methods. Funnyeye and Paperglasses remain difficult, with ACER of 24.37% and 20.64%.

The MSU-MFSD, CASIA-MFSD, and Replay-Attack experiments also leave out one attack type. The method reaches the table's highest mean AUC, 98.0 ± 4.26% across nine tests. This is cross-type evaluation, rather than evaluation on an entirely unseen dataset. MSU print attacks reach 86.78% AUC, above the listed methods but the weakest of this method's nine results.

The average supports the dual-stage approach.

Same-type Experimental Performance

SiW’s three protocols cover pose/expression, presentation media, and attack types. The original section title is retained, but Protocol 3 is a cross-type test:

  1. Protocols 1 and 2: Both achieve 0% ACER. Protocol 1 ties DST, while Protocol 2 ties DST and LGC.
  2. Protocol 3: ACER is 3.58 ± 3.93%, worse than LGC’s 1.19 ± 1.39% and HMP’s 2.45 ± 0.68%. The authors suggest that training on only one attack type may constrain disentanglement; this is a proposed explanation, not an isolated causal result.

Results on Protocols 1 and 2 should be assessed separately from transfer with a single training attack type.

Visualization of Disentangled Features

To illustrate the disentangled features, the authors performed a “face swap” experiment.

They preserve each image’s live feature and exchange the two images’ spoof features, producing Drec(FLA,FSB)D_{rec}(F_L^A,F_S^B) and Drec(FLB,FSA)D_{rec}(F_L^B,F_S^A). They do not exchange a live feature with a spoof feature.

The results show that after swapping, the real face acquires spoof textures such as “stripes or grayscale imbalance,” while the original spoof face partially recovers natural skin tone.

The examples support the interpretation that spoof features carry some attack texture.

Spoof Map Analysis

Finally, to observe what the disentangled spoof feature looks like, the authors decoded it via a U-Net style decoder DmapD_{map} to generate detection maps.

In the figure's genuine-face examples, the spoof maps are nearly zero. In its print-attack examples, bright regions appear around the forehead, eyes, nose, and mouth.

These maps provide visible image-space cues; whether each highlighted area corresponds to a physical attack mechanism requires separate validation.

Ablation Study

Because DualStage employs many loss functions, the authors conducted ablation experiments on their effects and compared results with other methods.

Removing the first stage or the adversarial loss reduces mean AUC across the nine tests from 98.0% to 95.01% or 94.0%, respectively. Removing all triplet loss gives 95.01%; retaining only normal triplet loss gives 97.25%.

These ablations support contributions from genuine-face pretraining, adversarial loss, and both triplet losses in the tested setting. Removing the auxiliary classifier also brings mean AUC down to 93.3%.

Conclusion

DualStage proposes a straightforward design:

Split the disentanglement process into two stages — first stably learn liveness features, then gradually peel off spoof signals.

It needs no extra depth maps or rPPG supervision. From the images themselves, it produces spoof maps useful for classification and shows strong results on several public cross-type tests. That gives the method a place even without support from multiple sensing modalities.

Of course, it still has drawbacks: performance is less satisfactory in scenarios with extremely sparse attack types (e.g., SiW Protocol 3). Future improvements may require further handling of seemingly “irrelevant” but actually disruptive feature fragments.

As a methodological proposal, it leaves an important message:

Disentanglement does not have to be done all at once.

The model can learn gradually, as long as it is given the right sequence.