Back to paper library
arXiv 2021IEEE TCSVT 2022Face anti-spoofingPaper notes

DSDG

The Eve of Illusion Recombination

Original paper

Dual Spoof Disentanglement Generation for Face Anti-spoofing with Depth Uncertainty Learning

Paper authors

Hangtong Wu · Dan Zen · Yibo Hu · Hailin Shi · Tao Mei

First submitted to arXiv
arXiv:2112.00568
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@misc{faceantispoofingdsdg2021,
  title = {{Dual Spoof Disentanglement Generation for Face Anti-spoofing with Depth Uncertainty Learning}},
  author = {Hangtong Wu and Dan Zen and Yibo Hu and Hailin Shi and Tao Mei},
  year = {2021},
  eprint = {2112.00568},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2112.00568}
}
On this page

Some illusions have never appeared.

Not before you recognized them.

Problem Definition

When face recognition systems begin to bear responsibility for real-world verification, accuracy is no longer just a number on a leaderboard, but a boundary of risk.

Face Anti-Spoofing (FAS), as the gatekeeper of this boundary, has in recent years evolved from a binary classification task into a contest of generalization across diverse conditions.

However, most existing methods remain trapped by an unresolved assumption:

The limitation of data leads to models generally lacking the ability to recognize unknown identities and attack types.

The training datasets we use mostly come from closed environments, fixed devices, few identities, and limited attack types.

For example, the OULU-NPU and SiW training splits have only 20 and 90 subjects. Such data distributions cannot cover the range of devices, populations, complex lighting, and attack techniques that models may meet in actual deployment. The result is that a model can overfit its training samples and fail to recognize attacks under unseen conditions.

Searching for more and more data has become the default solution.

If we consider image translation methods, most can only "swap faces but not identities," so generated samples remain bound to the original identity; synthetic processes based on external data collection face practical costs such as copyright, annotation, manpower, and delay.

Furthermore, when we attempt to introduce generative models to solve the data bottleneck, we inevitably encounter another problem:

The quality of synthetic samples varies, which may disrupt the stability of supervisory signals.

Especially in a framework supervised by depth maps, blurred and distorted images can undermine accurate depth prediction, affecting the stability and interpretability of the training process.

This is a dual-problem structure:

  • How to generate synthetic images that simultaneously possess new identities and existing attack patterns, improving intrinsic data variability?
  • How to ensure these generated samples do not negatively pollute depth-supervised training, causing the main network to learn incorrect visual cues?

The authors propose a response on two fronts: how synthetic data are generated, and how the model is trained with them.

Before moving on to the next section, let us refocus on the original question:

How many illusions must a truly useful FAS system have seen to confidently say, "This is not real"?


Solution

To address the two major problems of "data scarcity" and "image distortion," the authors integrate two ideas: "generative data augmentation" and "uncertainty learning," establishing a dual-module architecture:

  • DSDG (Dual Spoof Disentanglement Generation)
  • DUL (Depth Uncertainty Learning)

Dual Spoof Disentanglement Generation (DSDG)

DSDG aims to learn a generator using the existing training data alone, then synthesize varied, paired live and attack images.

The authors use a dual-encoder Variational Autoencoder (VAE) to disentangle facial images into "identity features" and "attack features," then recombine them in the decoder to generate new paired images.

This approach simultaneously expands the coverage of identities and attack modes.

As shown in Figure (a), the overall structure consists of two encoders and one decoder, plus a disentanglement module that distinguishes spoof and identity features. The specific steps are:

  1. Dual Encoder Feature Outputs

    • Encs\text{Enc}_s inputs the attack image IspoofI_{\text{spoof}}, separately extracting the "attack pattern feature" zstz^t_s and the "identity feature" zsiz^i_s.
    • Encl\text{Enc}_l maps the live image IliveI_{\text{live}} to the identity feature zliz^i_l;

    Using the reparameterization trick, each encoder outputs the mean μ\mu and standard deviation σ\sigma, then obtains features by

    z=μ+ϵσ,ϵ∼N(0,I)z = \mu + \epsilon \sigma,\quad \epsilon \sim \mathcal{N}(0, I)
  1. Reconstruction and Disentanglement

    The decoder Dec\text{Dec} receives the three features (zst,zsi,zli)(z^t_s, z^i_s, z^i_l) and generates reconstructed I^spoof\hat{I}_{\text{spoof}} and I^live\hat{I}_{\text{live}} respectively.

    To ensure the attack feature is correctly extracted, a spoof classifier is attached on zstz^t_s with a cross-entropy loss (LclsL_{\text{cls}}) to enforce correct attack-type prediction.

    The authors also introduce an angular orthogonality constraint (LortL_{\text{ort}}) between the attack feature (zst)(z^t_s) and identity feature (zsi)(z^i_s), encouraging their vectors to be perpendicular rather than confused with one another.

  2. Distribution Alignment and Reconstruction Loss

    To achieve VAE feature distribution learning, the authors apply Kullback-Leibler (KL) divergence (LklL_{\text{kl}}) to force the encoded distributions toward a standard Gaussian.

    LrecL_{\text{rec}} uses L1 distance between each reconstructed genuine/attack image and its input.

    MMD (LmmdL_{\text{mmd}}) aligns the distributions of the two latent identity representations, while LpairL_{\text{pair}} compares the reconstructed pair's identity features through a pretrained LightCNN to promote identity consistency.

All losses are combined into the total objective function:

L=Lkl+Lrec+λ1Lmmd+λ2Lpair+λ3Lort+λ4Lcls\mathcal{L} = L_{\text{kl}} + L_{\text{rec}} + \lambda_1 L_{\text{mmd}} + \lambda_2 L_{\text{pair}} + \lambda_3 L_{\text{ort}} + \lambda_4 L_{\text{cls}}

After training the VAE-like structure, as shown in Figure (b), multiple pairs (z^st,z^si)(\hat{z}^t_s, \hat{z}^i_s) can be generated by sampling from a standard Gaussian noise. Then z^si\hat{z}^i_s is copied as z^li\hat{z}^i_l (so the pair shares an identity input) and passed to the decoder to generate new samples (I^spoof,I^live)(\hat{I}_{\text{spoof}}, \hat{I}_{\text{live}}).

This generation step needs no external data, although training the DSDG spoof classifier still uses attack-type labels from the original data. The generated samples can show new identities and attack patterns learned from that data.

However, VAEs can produce blurry or locally distorted images. Adding these noisy samples directly to training may confuse depth supervision.

To address this, the authors propose Depth Uncertainty Learning as a remedy.

Depth Uncertainty Learning (DUL)

Beyond the diversity of synthetic samples themselves, the authors further address the interference problem in depth supervision by introducing an uncertainty representation mechanism that quantifies and suppresses the negative effects of generative noise via Gaussian distributions.

Traditional depth-supervised methods produce a fixed depth value for each image region. When a synthetic image has local distortions, however, its depth supervision may be unreliable.

The authors treat the depth value as a Gaussian distribution N(μi,j,σi,j2)\mathcal{N}(\mu_{i,j},\sigma_{i,j}^2), where μi,j\mu_{i,j} is the learnable depth mean and σi,j\sigma_{i,j} represents uncertainty.

During training, the value actually involved in the loss calculation is a sampled depth di,j=μi,j+ϵσi,jd_{i,j} = \mu_{i,j} + \epsilon \sigma_{i,j}, where ϵ∼N(0,I)\epsilon \sim \mathcal{N}(0, I).

This "reparameterization" approach allows uncertainty to be learned through backpropagation.

Within this framework, the model applies two convolutional operations to the depth feature map: one outputs μi,j\mu_{i,j}, the other σi,j\sigma_{i,j}.

KL divergence regularizes N(μi,j,σi,j2)\mathcal{N}(\mu_{i,j},\sigma_{i,j}^2) toward N(μ^i,j,I)\mathcal{N}(\hat{\mu}_{i,j},I), whose mean is the supervisory depth value and whose variance is one. This is distributional regularization, not an explicit reduction of the loss weight for each uncertain pixel.

The authors attach the module to depth-supervised networks including ResNet, MobileNetV2, DepthNet, and CDCN, yielding pixel-level mean and standard-deviation maps; inference uses the mean depth map.

The authors use mean squared error (LMSEL_{\text{MSE}}) as the main depth supervision and add a weighted term λklLkl\lambda_{\text{kl}} L_{\text{kl}} in the total loss to control the strength of uncertainty regularization.

Overall, the total loss can be expressed as:

Loverall=LMSE+λklLkl+λg(LMSE′+λklLkl′),L_{\text{overall}} = L_{\text{MSE}} + \lambda_{\text{kl}} L_{\text{kl}} + \lambda_g \Bigl(L_{\text{MSE}}' + \lambda_{\text{kl}} L_{\text{kl}}'\Bigr),

Here λg\lambda_g controls the weight of the generated-data loss in backpropagation. A separate parameter rr controls the fraction of original samples in each batch.

Through these two modules, the authors combine generated-data augmentation and depth uncertainty learning in the FAS training process.

Discussion

Through ablations, within-dataset and cross-dataset experiments, and visualizations, the authors assess DSDG's data augmentation and DUM's depth uncertainty learning. The results of each protocol still need to be read separately.

Ablation Studies

The authors ablate key hyperparameters and modules, including the effect of the number of identities, the generated-image ratio (through rr), and the contribution of losses such as LmmdL_{\text{mmd}}, LpairL_{\text{pair}}, and LortL_{\text{ort}} to identity disentanglement.

  1. Number of Identities and Generalization Performance

    On OULU-NPU Protocol 1 (P1), as training identities increase from 5 to 20, this method's ACER falls from 5.1% to 0.3%; the CDCN baseline also falls from 8.3% to 1.0%. Both are affected by the number of training identities, and this method is lower at every count shown.

  2. Synthetic Image Ratio rr

    The authors vary rr, the fraction of original samples in a training batch. On OULU-NPU P1, r=0.75r=0.75 gives 0.3% ACER, the lowest in the table.

    Increasing the number of generated image pairs from 10k to 30k gives the same 0.3% ACER at 20k, 25k, and 30k. The authors subsequently use 20k pairs; the table does not show that more samples add noise.

  3. Importance of Identity Disentanglement Losses

    On OULU-NPU P1, the complete method has 0.3% ACER. Removing LmmdL_{\text{mmd}}, LpairL_{\text{pair}}, or LortL_{\text{ort}} separately gives 1.0%, 0.7%, and 0.7%, respectively.

    These results support a contribution from all three losses under this protocol.

  4. Hyperparameters λkl\lambda_{\text{kl}} and λg\lambda_g

    The authors test sensitivity to the KL weight (λkl\lambda_{\text{kl}}) and generated-data loss weight (λg\lambda_g). Most tested settings beat the CDCN baseline of 1.0% ACER, but λg=0.01\lambda_g=0.01 gives 1.4%; the method is therefore not uniformly insensitive to these parameters.

Intra-Dataset Testing

To evaluate recognition ability within the same dataset, the authors select OULU-NPU and SiW.

  • OULU-NPU (Table X): ACER across the four protocols is 0.3%, 1.2%, 1.4%, and 2.3%, the lowest values shown in the table. Protocol 4 covers changes in environment, attack, and input device.
  • SiW (Table XI): ACER on the first two protocols is 0%, tied with several methods in the table. Protocol 3 reaches 3.80% ± 1.44%, higher than CDCN's 1.71% ± 0.11%.

Together, the results support good performance across OULU-NPU's four protocols, but SiW Protocol 3 is a clear exception. Stability across every attack type cannot be inferred.

Model Analysis and Visualization

The authors use various visualization methods to show the nature of generated images, attack feature disentanglement, and depth uncertainty distribution, helping readers understand DSDG and DUM’s internal workings.

  1. Generated Images

    The figure places original data beside generated live/attack pairs for different identities and attack patterns. It illustrates variety among the generated images, but cannot by itself verify identity consistency.

    On the attack-diverse SiW-M dataset, the authors also show generated Replay, Print, 3D Mask, Makeup, and Partial Attack examples. These images cannot quantify the amount of distortion.

  2. Attack Pattern Disentanglement

    The image above is the t-SNE projection in Figure 7 of the paper. Without the attack-type classifier and orthogonality loss (LortL_{\text{ort}}), the attack representations are more mixed; after training the complete DSDG, the projection shows clearer clusters for the four classes. Both constraints change together in this comparison, so the figure cannot isolate the effect of either one.

  3. Depth Uncertainty

    This image corresponds to Figure 9 in the paper. It compares depth maps from CDCN and the method with DUM, alongside predicted standard deviations. The authors note higher standard deviation around reflections, face edges, and the background. These are qualitative examples; the image cannot show that the model directly reduces the loss weights of these pixels.

Conclusion

DSDG answers a simple but persistent question:

Without extra annotations or external data, how can we expand identity and attack diversity while limiting the damage synthetic noise does to depth supervision?

When this paper appeared, the full surge of deepfakes and the widespread use of diffusion models still lay ahead. The authors had already realized that "data problems cannot be solved by data alone," and tried to advance the boundary of generalization through two modular strategies: generation and uncertainty.

Of course, limitations remain:

  • Using VAE as the generative backbone restricts resolution and detail stability;
  • Dependence on depth limits extension to non-depth-supervised scenarios;
  • Dependence on spoof types remains and is not fully relaxed, impacting purely unsupervised settings.

Nevertheless, these do not diminish its role in the technology chain. DSDG reminds us:

Data diversity, supervision reliability, and model robustness must be considered simultaneously, or reinforcement only stacks noise.

Models evolve, attacks mutate, and algorithms are forgotten. But those who tried to take the problem apart, rather than chase scores alone, leave traces that become footholds for those who follow.