Cyclically Disentangled Feature Translation for Face Anti-spoofing
- Publication
- AAAI 2023Conference
- First submitted to arXiv
- arXiv:2212.03651
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{faceantispoofingcdftn2022,
title = {{Cyclically Disentangled Feature Translation for Face Anti-spoofing}},
author = {Haixiao Yue and Keyao Wang and Guosheng Zhang and Haocheng Feng and Junyu Han and Errui Ding and Jingdong Wang},
year = {2022},
eprint = {2212.03651},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2212.03651}
}
After SSAN brought the idea of "style" into our FAS reading, many researchers began to focus on this question.
This led to several follow-up studies, and we continue to explore this paper.
Defining the Problem
FAS may be posed as binary classification, but in practice it is a gate at which a live face must be distinguished from an attack.
On one side, there is a real human face, with flesh and breath; on the other, there is a high-resolution photo, a replayed video, or a silicone mask.
Early methods relied on handcrafted features, such as LBP, HOG, and SIFT, which once shone brightly. Later, CNNs took over, starting with ImageNet pre-trained models like VGG and ResNet, progressively stacking deeper and wider architectures. Some methods began incorporating LSTMs to capture temporal information, and others introduced a materials science perspective.
But no matter how methods evolved, one problem remained unsolved:
Change the dataset, and the model becomes useless.
When the shooting device, ambient lighting, or cultural region changes, liveness models become like a disrupted compass, unable to point in the right direction.
Thus, Domain Generalization (DG) was born. We want models to learn the abstract "liveness semantics" rather than memorize the appearance of a specific dataset. However, DG methods are often built on multiple labeled domains, a resource that is not commonly available in real-world scenarios.
A more deployment-friendly approach is Unsupervised Domain Adaptation (UDA), where there is a labeled source and an unlabeled target.
UDA methods typically fall into three categories:
- Feature alignment, minimizing the distribution gap between source and target;
- Adversarial training, using a discriminator to obscure the source;
- Image transformation, directly making the source look like the target.
All are effective but have one problem:
They don't "truly disentangle semantics and style."
The authors’ motivation is that distribution alignment or image translation may fail to preserve information needed for live/spoof decisions. These three techniques can also overlap.
So, if we accept that the difference between live and spoofed faces comes from "semantics," and the difference between domains comes from "style and background," can we design a model that truly overturns the domain framework?
Solving the Problem
This paper introduces an architecture called Cyclically Disentangled Feature Translation Network (CDFTN for short).
You can probably sense the intense pressure from the above diagram—it looks complex, and it truly is.
The authors’ goal is not to directly classify real vs fake faces, but to first "disentangle" the latent features of a face across different dimensions, and then "reassemble" them into a representation that can bridge domain differences.
The entire process can be divided into two phases:
- Phase 1: Feature disentangling, cross-domain translation, and generation of new images.
- Phase 2: Using the images generated in Phase 1 to train the final classifier.
The core concept of this method is to decompose each face into two subspaces:
- Liveness Space: Corresponds to semantic features of live vs. spoofed faces (which should exhibit cross-domain stability).
- Content Space: Contains domain-specific information, such as device, lighting, and background, which can be seen as interference or noise.
Next, you’ll need some background in adversarial learning, so let’s assume you’re already a GAN master. Let’s dive straight in.
Phase 1
Feature Disentangling and Adversarial Learning
Each domain has a liveness encoder and a content encoder:
- : Extract liveness features from the source and target domains, respectively.
- : Extract content and style features from the source and target domains, respectively.
To keep liveness features focused on the live/spoof decision while hiding which domain they came from, the authors use adversarial learning to encourage both liveness encoders to produce domain-indistinguishable latent vectors . Its loss function can be written as:
Here, distinguishes whether comes from the source or target; the two liveness encoders try to "trick" it so their extracted features are difficult to distinguish by domain.
Classifier for Retaining Liveness Information
A classifier supervises liveness discrimination on the source. The paper uses a two-element one-hot label ; with denoting the probability of class , the full cross-entropy is:
Source classification encourages discriminative features, while domain adversarial training encourages alignment. These are optimization objectives, not guarantees of perfect disentanglement or correct target-domain classification.
Cross-Domain Generation and Style Recombination
This is where CDFTN shines.
In the first phase, the model not only extracts liveness features but also needs to "exchange liveness and content features across different domains" to generate new "pseudo-labeled images."
The reason for this is that in real-world applications, the source and target domains often have significant differences (e.g., camera brand, shooting environment, lighting), and relying solely on data from the source domain for training usually fails to generalize to the target.
By using cross-domain recombination, the model can better "see" the potential variations in style across domains, thus expanding the available training data and making the final learned liveness detection more cross-domain stable.
First, we obtain the following from the source and target domains:
Here, represents the features most relevant for distinguishing real vs. fake faces in the "liveness space," while represents the domain-specific style information (such as device effects, background, lighting characteristics).
Next, the latent vectors of the two domains are exchanged, generating the following "pseudo-labeled images":
Here are separate domain decoders. The target-style synthetic image inherits the source label through its source liveness feature. This transfers a label to a generated sample; it does not assign a predicted label to an original unlabeled target image. The reverse image supports the cycle constraints and does not automatically have a known target ground-truth label.
The significance of this is: if we can preserve the original real/fake semantics (carried by ) while simultaneously making the style closer to the target domain (determined by ), we can obtain additional training images that "look similar to the target but still preserve real or fake face semantics." These images can then be used by the classifier to learn better recognition in the target domain.
Separate image discriminators distinguish captured images from generated ones in each domain. GAN real/fake here means captured/generated, not FAS live/spoof. The adversarial objective includes both domains:
The generators try to make their images look like real samples from the corresponding domains so that the discriminators cannot identify them as generated. That is the training objective; the authors also report that the generated images do not have good visual quality.
Consistency Loss
To avoid confusion or loss of liveness features or original image content during the style transformation process, the authors introduce three additional consistency losses to strengthen constraints, ensuring that the exchanged images retain their "real/fake" semantics and do not suffer from excessive style distortions.
-
Reconstruction Loss
Without cross-domain exchange (i.e., sending the same image's and back to the same decoder), we should be able to reconstruct the original image. If the reconstruction error is too high, it indicates that the encoder or decoder failed to correctly extract or restore the image, and it needs to be penalized during training.
-
Cycle Loss
Inspired by the concept of CycleGAN, if we have generated (the exchanged image), we can feed it back into the corresponding encoder to extract and then convert it back to the original domain. We require the final reconstruction to be "as close as possible" to the original image. This "round-trip" mechanism helps prevent irreversible information loss during cross-domain transformations.
-
Latent Space Loss
This loss primarily ensures that the liveness features do not undergo significant deformation after cross-domain exchange. For instance, if the original image represents a "real face," after multiple domain transformations, it should still maintain the "real face" semantics, not be distorted by style exchanges or noise operations. This loss constrains the model to preserve the semantic consistency of liveness features before and after the exchange.
Combining these steps creates a closed-loop feature translation process: disentangle, exchange, then use adversarial learning and consistency losses to correct the result. The target-style pseudo-labeled images carry source liveness labels and will be used in the second phase.
Phase 2
After completing Phase 1, the authors fix the entire generation framework and use the "pseudo-labeled images" generated earlier to train the final classifier . The paper compares two design choices:
- LGSC (Feng et al., 2020): A multi-task design with classification, regression of spoof cues, and triplet loss.
- ResNet-18: A binary classification design.
If using LGSC, the final loss can be written as:
Whereas if using ResNet-18, only the binary classification part is retained.
This is one of the more complex papers we've read recently. Overall, the core idea of CDFTN is:
- First disentangle liveness semantics from domain-related content, using adversarial learning to encourage cross-domain consistency of liveness features.
- Then exchange features between domains, generating more "pseudo-labeled images" through cross-domain generation.
- Finally, use these pseudo-images as supplemental data to train a more robust final classifier.
From a methodological perspective, this architecture is indeed complex, but its approach is straightforward: the "real/fake" judgment of the same face should not be influenced by differences like camera brand or lighting conditions. Therefore, by first disentangling the "liveness" semantics and then attributing domain differences to content space, we can achieve more generalized and accurate liveness detection across domains.
Discussion
This paper evaluates the effectiveness of CDFTN in the context of cross-domain face anti-spoofing through both Single Target Domain Translation (SS2ST) and Multi-Target Domain Translation (SS2MT).
The authors used four commonly used cross-domain FAS datasets:
- CASIA-MFSD (C)
- Replay-Attack (I)
- MSU-MFSD (M)
- Oulu-NPU (O)
In the single target domain setting, one source (including all data) and one target (with its training set) are selected for the experiment.
This results in 12 test scenarios:
- C → I, C → M, C → O
- I → C, I → M, I → O
- M → C, M → I, M → O
- O → C, O → I, O → M
In the multi-target domain setting, one of the C, I, M, O datasets is chosen as the source, and the other three are treated as unlabeled targets.
Thus, there are 4 scenarios:
- C → I & M & O
- I → C & M & O
- M → C & I & O
- O → C & I & M
For evaluation metrics, the authors used the following two common quantitative metrics:
-
HTER (Half Total Error Rate):
where and are the False Rejection Rate and False Acceptance Rate, respectively.
-
AUC (Area Under the Curve): The higher the AUC, the better the classifier's ability to distinguish between real and fake faces.
Comparison with Other Methods
The authors first compared CDFTN with other cross-domain learning methods, and the results are shown in the table above:
- Table 1 reports HTER, not per-direction AUC. CDFTN-L averages 13.2% across 12 directions, and CDFTN-R averages 13.8%. Baseline averages with missing directions should not be compared directly with these complete averages.
- For C → O, CDFTN-L/R scores 29.9%/32.5% HTER, worse than MDDR at 24.7% and some other baselines. Nor is each CDFTN variant best elsewhere: CDFTN-R has 14.4% on C → M versus DR-UDA’s 9.0%. The proposed explanations involving Oulu’s attack devices and resizing blur were not isolated in a controlled causal experiment.
CDFTN is itself a UDA method. These comparisons support disentanglement, generation, and classifier retraining in several settings; they do not show that domain adaptation generally fails or prove universally invariant liveness representations.
Multi-Target Domain Feature Translation
The authors compared three different settings:
- SS2ST: Single target domain.
- SS2BT: Merging multiple target domains into a "mixed" single domain for SS2ST.
- SS2MT: Uses separate domain encoders, decoders, and associated discriminators, cycling liveness features from the source through the targets and back. Source liveness features and each target’s content then generate training images.
For all four groups of target-averaged HTER and AUC values in Table 2, SS2MT outperforms both SS2BT and SS2ST.
This indicates that when dealing with multiple subdomains, using a single encoder (treating all targets as one entity) may degrade performance due to large internal domain differences.
SS2MT uses multiple unlabeled target domains in a joint training process before training a classifier for them. This avoids separately running the whole pipeline per target, but subnetworks grow with the number of domains, so compute, memory, and elapsed time need measurement. Table 2 also reports target-averaged results, not improvement in every individual target.
Ablation Study
The authors remove four loss terms in turn. Table 3 above reports CDFTN-L with LGSC; the ResNet-18/LGSC comparison is primarily in Table 1.
The results show:
- Removing (the cross-entropy loss for the liveness classifier on the source) caused a performance drop, indicating that if doesn't retain liveness discriminability, the model struggles to extract genuinely discriminative liveness features.
- Removing (cycle consistency) removes the closed-loop translation constraint; mean HTER rises from 8.1% to 11.0%.
- Removing (self-reconstruction loss) caused the model to lose the ability to accurately restore the original image, undermining the basic constraints on the encoder/decoder. This had a significant impact on performance.
- Removing (latent space consistency) caused liveness features to distort or be lost during reconstruction, similarly degrading results.
Across the nine directions in Table 3, full-model HTER averages 8.1%. Removing reconstruction, liveness classification, latent consistency, and cycle consistency gives 19.7%, 12.0%, 11.7%, and 11.0%, respectively. Reconstruction has the largest effect, but latent consistency is not second by these averages, despite that ranking in the paper’s prose. Ablations support usefulness in this setting rather than directly proving the proposed mechanisms.
Visualization Analysis
For the M → I experiment, the authors sample 500 examples from each domain and use t-SNE to display the features in 2D. Blue and orange denote source and target; circles and crosses denote live and spoof faces. Before translation is on the left and after translation on the right. In this example, same-class examples across domains are closer and the drawn decision boundary is clearer after translation. The plot alone cannot quantify domain differences in every setting.
Conclusion
In the research history of FAS, the weight of the term "style" has gradually increased.
Initially considered an interference factor, it later became a key explanation for the failure of recognition generalization, and further became an explicit modeling dimension. The relationship between style and semantics has become the core of many methodologies in recent years.
CDFTN is a key point along this path.
It not only disentangles semantics from style, but more importantly, it transforms this disentanglement into part of the generation mechanism and trains a classifier using cross-domain pseudo-labeled samples, attempting to establish a semantic-driven style reconstruction process.
Compared to research directions focusing on "how to combat domain shift," respecting the existence of style and allowing the model to learn to coexist with it is also an important topic.