Face Anti-Spoofing Via Disentangled Representation Learning
- Publication
- ECCV 2020Conference
- First submitted to arXiv
- arXiv:2008.08250
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{faceantispoofingdisentanglefas2020,
title = {{Face Anti-Spoofing Via Disentangled Representation Learning}},
author = {Ke-Yue Zhang and Taiping Yao and Jian Zhang and Ying Tai and Shouhong Ding and Jilin Li and Feiyue Huang and Haichuan Song and Lizhuang Ma},
year = {2020},
eprint = {2008.08250},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2008.08250}
}
This paper did not give itself a catchy name.
So we simply adopt the paper’s title and call it: Disentangle-FAS.
Problem Definition
Face Anti-Spoofing (FAS) has always stood at the forefront of recognition systems.
From unlocking phones, access control, financial identity verification, to various eKYC scenarios, we rely on this task to tell whether a face is a real person or some kind of disguise.
As algorithms improve, datasets expand, and model accuracy climbs, we thought the problem was being solved.
But in fact, we have just used more varied disguises to make the model memorize more answers.
Looking closely, the entire field is actually stuck on several persistent problems:
- Black-box models for real-vs-fake classification that cannot explain their decisions;
- Extremely poor generalization—models collapse once the dataset changes;
- Data is never enough, and attack methods evolve even faster.
More seriously: we cannot even clearly describe what “liveness features” actually are.
This is not merely a matter of choosing the wrong architecture or collecting too little data.
Rather, our understanding of the face itself is too chaotic.
Too many semantics coexist on the face: identity, pose, lighting, background, device characteristics, skin texture, etc., yet we force the model to make a one-shot “real or fake” decision in this tangled syntax.
Is this approach fundamentally flawed from the start? Could it be possible that:
We let the model first try to extract an independent clue belonging to “liveness” from the entanglement of noise and semantics?
Solution
To untangle the “mixed semantics on the face,” the authors propose two core components:
- A Disentanglement Process that separates liveness and content.
- Multiple forms of Auxiliary Supervision (such as LBP, Depth, and Discriminators) to strengthen the disentanglement effect.
Disentanglement Process
First, the authors design an “exchange and recombination” pipeline that splits the latent feature vectors of face images into two parts:
- Liveness Features (): representing the key factors of “whether the face is real or fake.”
- Content Features (): intended to retain identity, background, lighting, and pose—the attributes unrelated to whether the face is live or spoofed.
The full process is illustrated in the figure above:
-
A real face and a fake face are separately fed into the encoder, yielding and .
-
Then, the liveness features are swapped to synthesize new images and , which respectively mean:
- Image reconstructed by combining “fake face’s liveness features” with “real face’s content features.”
- Image reconstructed by combining “real face’s liveness features” with “fake face’s content features.”
In other words, combining the live face's with the spoof face's should produce an image with B's content but A's live attributes; the reverse combination should carry A's content and B's spoof attributes.
The swap is meant to make the model separate "real or fake" from facial content. If the two remain mixed, a swapped image may look distorted or retain the wrong traces; the losses encourage separation without guaranteeing complete independence.
Reconstruction Loss: Image & Latent
The Encoder + Decoder combination is constrained to reconstruct the image.
The authors impose two types of reconstruction losses as constraints:
-
Image Reconstruction Loss
They expect to approximate the original itself, defined as:
where is the encoder, is the decoder, and is the distribution of real images.
-
Latent Reconstruction Loss
The swapped combination should be decoded into an image and re-encoded to recover that same :
This constrains the swapped liveness and content features to survive decoding and re-encoding, encouraging their separation.
Auxiliary Supervision
To further “lock down” the liveness features, the authors add three auxiliary supervision signals within the disentanglement process:
- LBP Map
- Depth Map
- Discriminative Supervision (Discriminator)
The table below shows the detailed network architectures for each auxiliary module:
Each network consists of a series of convolutional layers, batch normalization (BN), and ReLU activations, using features at different levels for calibration or discrimination.
-
LBP Map
Considering that “fake faces” often have texture flaws (e.g., printing dots, moiré patterns on screens), the authors apply LBP (Local Binary Pattern) supervision on the liveness features:
- Real faces: expect the LBP network output to approximate the true LBP map;
- Fake faces: receive an all-zero target map. This is a training label, not a claim that computing LBP directly on an attack image produces zeros or that it lacks texture.
The loss function is formulated as:
where denotes the real face distribution, the fake face distribution, the real face’s LBP image, and is the all-zero map.
-
Depth Map
For the 3D geometric information of real faces, the authors also introduce depth supervision:
- Real faces correspond to a “pseudo depth map” generated by 3D alignment algorithms;
- Fake faces correspond to an all-zero map.
The loss function is:
is a pretrained depth network whose parameters remain fixed during disentanglement training. Gradients can pass through it to generated images without updating its weights. Live targets are algorithmic pseudo-depth; zero attack targets are a training convention.
Reconstructed and swapped images receive pseudo-depth or zero-map supervision according to their intended live/spoof status, encouraging the branch to control this attribute.
-
Discriminative Supervision
Finally, the authors design multi-scale discriminators to help determine whether the “generated images” look realistic.
Essentially a GAN-style adversarial training:
- Discriminator loss (): update the discriminator with the generator fixed to learn to distinguish real (R) from generated (G);
- Generator loss (): update the entire Disentanglement Network with the discriminator fixed to force it to produce more realistic images.
The generator loss is defined as:
Where and are discriminators at different scales capturing local details and global structure.
Final Objective Function
The authors sum all the above losses to form the final training objective:
Here, are important hyperparameters balancing each loss term.
The entire training adopts a typical adversarial iterative scheme: first optimize the discriminator, then optimize the entire disentanglement network.
Back and forth, the goal is to achieve:
- Synthesized images realistic enough after swapping;
- Liveness and content features as separate as possible while remaining reconstructable;
- LBP and depth supervision solidifying that liveness features indeed carry real/fake information.
At inference, there is no reconstruction or swapping. The model averages the score from the LBP map estimated from and the score from the depth map estimated from the image to decide whether the face is live. Cross-domain ability must still be checked experimentally.
Discussion
The authors conducted experiments on four common FAS datasets:
- Oulu-NPU and SiW (used for intra-testing)
- CASIA-MFSD and Replay-Attack (used for cross-testing)
They evaluated performance using standard metrics: APCER, BPCER, ACER, and HTER.
Intra-testing: Oulu-NPU
On OULU-NPU, Protocol 1 ACER is 1.3%, tied with FAS-TD. Protocol 2 yields 2.4%, above FAS-TD’s 1.9%. Protocols 3 and 4 yield the lowest mean ACER values in the table, 2.2% and 4.4%, respectively. These results support performance under the tested protocols; the table alone cannot attribute that performance to feature disentanglement.
Cross-testing: CASIA-MFSD ↔ Replay-Attack
The cross-testing design trains on CASIA and tests on Replay, or vice versa, which is a standard “cross-domain” evaluation for FAS.
For CASIA → Replay, HTER falls from the table’s prior best 23.6% to 22.4%, a 1.2 percentage-point improvement. Replay → CASIA yields 30.3%, close to BASN’s 29.9% but worse than Auxiliary’s 28.4%. The advantage is asymmetric.
The two directions differ: cross-dataset performance still depends on which dataset is used for training and which for testing.
Translation Results and Visualization
The authors show three groups of translation results: “real ↔ fake,” “real ↔ real,” and “fake ↔ fake.”
- In the shown real ↔ fake swaps, predicted depth changes as well, supporting the presence of class-related information in the liveness features;
- For “real ↔ real” or “fake ↔ fake” internal swaps, the depth maps remain almost unchanged.
- These visualizations are consistent with the design goal of keeping identity, lighting, and background in the content features.
The authors also zoom in on local areas, such as print textures and screen moiré patterns, to show how the swapped images depict different spoof details:
These local examples show that the swapped images can depict some texture details of different attack media.
Ablation Study
The authors tested many ablation combinations, with the core conclusions:
-
With vs. Without Disentanglement: the t-SNE plots show live and attack samples more clearly separated in the liveness features after disentanglement, while the content-feature projection is more mixed.
-
LBP + Depth Supervision: in the OULU-NPU Protocol 1 ablation, combining low-level texture (LBP) and depth supervision gives the table's lowest ACER, 1.25% with averaged scores.
-
Score Fusion: in the same protocol, averaging the LBP and depth scores gives 1.25% ACER, below LBP alone (1.56%), depth alone (1.67%), or taking their maximum (1.88%).
The experiments and visualizations support the usefulness of liveness features for distinguishing real from fake and show some attack textures in the displayed examples. Cross-dataset results still depend on the test direction.
Conclusion
Disentangle-FAS does not attempt to build a more complex classifier. Instead, it rewrites the task by splitting the latent syntax of the “face” into two worlds:
One for liveness, one for content.
Within this framework, real or fake is no longer a scalar output score but a semantic space that can be swapped, generated, and visualized.
The model does more than distinguish faces: it can decompose them and show where a disguise may appear. Those visualizations help us inspect the result, although they cannot fully explain every decision.
Through visual translation of liveness features, auxiliary supervision from depth and texture, and cross-domain evaluations, this paper tells us:
Rather than just “judging real or fake,” we should teach models to “see where the fake originates.”
In the chaotic and ever-changing future of attacks, this path may not be easy, but it offers a chance to make defense syntactically meaningful, inferable, and explainable.
This marks a step away from the black box toward a more conscious form of visual recognition.