Back to paper library
arXiv 2023CVPR 2023Face anti-spoofingPaper notes

IADG

A Monologue of Styles

Original paper

Instance-Aware Domain Generalization for Face Anti-Spoofing

Paper authors

Qianyu Zhou · Ke-Yue Zhang · Taiping Yao …

Show all 7 authors

Qianyu Zhou · Ke-Yue Zhang · Taiping Yao · Xuequan Lu · Ran Yi · Shouhong Ding · Lizhuang Ma

Publication
CVPR 2023Conference
First submitted to arXiv
arXiv:2304.05640
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@misc{faceantispoofingiadg2023,
  title = {{Instance-Aware Domain Generalization for Face Anti-Spoofing}},
  author = {Qianyu Zhou and Ke-Yue Zhang and Taiping Yao and Xuequan Lu and Ran Yi and Shouhong Ding and Lizhuang Ma},
  year = {2023},
  eprint = {2304.05640},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2304.05640}
}
On this page

Over the past decade, researchers have tried every possible method to draw the boundary between the real and the fake:

  • Using handcrafted features like LBP, SIFT, and HOG to recognize textures and details;
  • Exploring anomalies in the YCrCb color space and Fourier spectrum;
  • Analyzing subtle differences in time-series, attempting to find evidence of a real face from blink, micro-movements, and blood flow;

Then, the whole field passed the baton to CNN, using classifiers, depth maps, reflectance maps, and rPPG for auxiliary supervision.

However, these methods often experience a significant drop in performance when the scene, device, or attack method changes, making them difficult to deploy.

Defining the Problem

To make models adaptable to unknown domains, researchers introduced the concepts of Domain Adaptation (DA) and Domain Generalization (DG).

In particular, in DG, adversarial learning and meta-learning became mainstream choices, aiming to learn a representation that says, "No matter where you come from, I can spot you."

But as this path progressed, its limitations began to emerge:

  • Many DG methods use explicit domain labels, although methods such as D²AM already avoid manually defined domain labels;
  • Dataset names, device models, and other human-defined groups are useful organizational labels, but may not capture fine variations in lighting and background.

You say "this is Replay," but Replay contains multiple lighting conditions and backgrounds; you say "this is Oulu," but no one can specify how many style subgroups it contains.

Even with labels, these methods only align features at the distribution level but fail to control for certain feature channels that still secretly retain style traces. These seemingly harmless signals often become blind spots for the model when faced with new samples.

How can we distinguish live faces from attacks without domain labels? Live/spoof labels are still required; this is not unsupervised learning.

Is there another boundary between real and fake?

Solving the Problem

To solve this problem, the authors propose a new architecture, called:

  • Instance-Aware Domain Generalization (IADG).

The goal is to enable the model to learn "instance-level" features without relying on domain labels, and to maintain stable performance in unknown domains.

It primarily consists of three core components:

  • Asymmetric Instance Adaptive Whitening (AIAW)
  • Categorical Style Assembly (CSA)
  • Dynamic Kernel Generator (DKG)

Let's take a closer look at the design of each module in order.

Dynamic Kernel Generator

Since samples from multiple source domains often have high diversity, a single static filter cannot meet the feature needs of all instances. To address this, the authors designed DKG to automatically generate "instance-adaptive" dynamic filters, working alongside static filters to capture more comprehensive individual perception features, aiding subsequent cross-domain generalization.

The structure of DKG consists of two parts: "static convolution branch" and "dynamic convolution branch":

  • The static branch shares its convolution parameters across samples; those parameters are still updated during training;
  • The dynamic branch generates convolution kernel parameters dynamically based on the features of each sample.

As shown in the figure above, let XiX_i and FiF_i represent the input and output features of the ii-th sample in DKG.

During training, both branches learn synchronously, and the process is as follows:

  1. The input feature XiX_i is split along the channel dimension into X^i\hat{X}_i and X~i\tilde{X}_i;
  2. Static convolution branch: X~i\tilde{X}_i is processed by a static convolution filter fθsf_{\theta_s} to produce Z~i\tilde{Z}_i;
  3. Dynamic convolution branch: X^i\hat{X}_i is first passed through global average pooling and a convolution block fθ1f_{\theta_1}, generating a dynamic convolution kernel WiW_i related to this instance. Then, this dynamic kernel is used to convolve X^i\hat{X}_i to obtain Z^i\hat{Z}_i.

Finally, Z~i\tilde{Z}_i and Z^i\hat{Z}_i are concatenated along channels and passed through the convolution block fθ2f_{\theta_2} to obtain FiF_i. This lets the static and dynamic filters work together for each instance.

Categorical Style Assembly

CSA generates more diverse instance-level style changes. It improves on earlier augmentation strategies that randomly swap or mix source styles; AdaIN itself neither requires random style selection nor determines class labels.

CSA makes two changes to how styles are selected for augmentation:

  1. Diversity Consideration: FPS iteratively selects styles far from those already selected, building LL basis styles and refreshing them each epoch. This encourages diversity, including less common styles, without guaranteeing coverage of every unseen distribution.

  2. Separate Treatment of Real and Fake: Styles are assembled within the same live/spoof class to reduce the risk of changing discriminative cues. Preserving the label is the design goal, not a general mathematical guarantee.

Each class contributes LL basis styles with stored channel mean and variance information. The assembly equations use means μbasec\mu_{\mathrm{base}}^c and standard deviations σbasec\sigma_{\mathrm{base}}^c; σ\sigma must not be confused with variance σ2\sigma^2.

For class cc, nonnegative weights summing to one are sampled from a Dirichlet distribution whose concentration parameters are all 1/L1/L. They combine basis means and standard deviations into μaug\mu_{\mathrm{aug}} and σaug\sigma_{\mathrm{aug}}, then transform a same-class sample as follows:

Faug=σaug(Forg−μ(Forg)σ(Forg))+μaugF_{\mathrm{aug}} = \sigma_{\mathrm{aug}} \Bigl( \frac{F_{\mathrm{org}} - \mu(F_{\mathrm{org}})}{\sigma(F_{\mathrm{org}})} \Bigr) + \mu_{\mathrm{aug}}

This expands style diversity while keeping the sample's original live/spoof label as the training target; the original/augmented feature pairs then help identify correlations sensitive to style changes.

Asymmetric Instance Adaptive Whitening

In order to "precisely align" the features of each sample, the author further focuses on the correlation between feature channels.

Previous studies have pointed out that the covariance matrix of features often contains style-related information. If whitening is applied to it, it can weaken some domain-specific styles.

However, simply applying existing instance whitening methods may result in the removal of "useful discriminative features," which could impair the performance of Face Anti-Spoofing (FAS).

The asymmetric design encourages compact live features while leaving more room for differences among attacks. Its actual procedure is:

  1. Apply IN separately to original and CSA-augmented features, flatten each C×H×WC\times H\times W map into C×HWC\times HW, and compute Σ=FF⊤/(HW)\Sigma=FF^\top/(HW).
  2. Compute the elementwise variance between the two covariance matrices for each sample, then average across samples to form VV. Large entries indicate sensitivity to the style perturbation.
  3. Rank entries to construct a binary mask M(kc)M(k_c), keeping only the strict upper triangle. Select a larger fraction for live samples, krk_r, than for attacks, ksk_s.
  4. Penalize the selected entries Σt⊙M(kc)\Sigma_t\odot M(k_c) in both original and augmented branches, pushing them toward zero.

AIAW is thus a selective decorrelation loss, not a layer that multiplies all features by an inverse covariance square root. The following whitening explanation provides background.

Overall Training and Optimization

To achieve stable performance in unknown domains, the classifier must be able to correctly distinguish between real and spoofed samples under various "style disturbances."

Therefore, for the same input sample XiX_i, the author supervises the binary classification loss LCls\mathcal{L}_{\mathrm{Cls}} using both the original feature ForgiF_{\mathrm{org}}^i and the augmented feature FaugiF_{\mathrm{aug}}^i:

LCls=−∑i∑t∈{org,aug}∑c∈{live,spoof}Yiclog⁡pict\mathcal{L}_{\mathrm{Cls}} =-\sum_i\sum_{t\in\{\mathrm{org},\mathrm{aug}\}}\sum_{c\in\{\mathrm{live},\mathrm{spoof}\}} Y_{ic}\log p_{ic}^{t}

This expands the paper’s shorthand into two-class cross-entropy: YicY_{ic} is a one-hot label, pictp_{ic}^{t} is branch tt’s probability for class cc, and ii ranges over source training samples. A single-sigmoid formulation requires both ylog⁡py\log p and (1−y)log⁡(1−p)(1-y)\log(1-p) terms.

Additionally, the author also references previous research and uses a depth estimator for additional supervision: outputting a depth map for real faces and a zero map for spoofed faces.

The depth loss function LDep\mathcal{L}_{\mathrm{Dep}} is:

LDep=∑i∑t∈{org,aug}∥Dep(Fti)−Ydepi∥22.\mathcal{L}_{\mathrm{Dep}} =\sum_i\sum_{t\in\{\mathrm{org},\mathrm{aug}\}} \left\|\mathrm{Dep}(F_t^i)-Y_{\mathrm{dep}}^i\right\|_2^2.

YdepiY_{\mathrm{dep}}^i is a supervision target: PRNet pseudo-depth for live faces and an all-zero map for attacks. It is not sensor-measured depth with the all-zero attack map serving as a training convention.

Combining the above, the final total loss function Ltotal\mathcal{L}_{\mathrm{total}} is:

Ltotal=LCls+λLDep+LAIAW.\mathcal{L}_{\mathrm{total}} = \mathcal{L}_{\mathrm{Cls}} + \lambda \mathcal{L}_{\mathrm{Dep}} + \mathcal{L}_{\mathrm{AIAW}}.

During the inference phase, only the original feature branch is used, and no augmentation operations are performed; the AIAW loss is only active during training.

DKG, CSA, and AIAW reduce sensitivity to style at the instance level. Generalization to a new environment still requires evaluation; the following results apply to the reported datasets and protocols.

Discussion

As usual, this section follows the traditional Leave-One-Out (LOO) experimental setup, followed by ablation experiments to analyze the contributions of each module.

Comparison with Existing Methods

Table 1 compares conventional FAS and DG methods. Domain-label granularity motivates the paper, but the table alone does not attribute every performance difference to those labels.

IADG has the lowest reported HTER in all four directions and the highest AUC in three. On OCM→I, its 94.50% AUC is slightly below SSDG-M’s 94.61% and SSAN’s 94.58%. It avoids domain labels, while still requiring live/spoof supervision.

Ablation Experiments

All ablations below use I&C&M→O; the values are not averages over four directions:

  • Using only the backbone (baseline) achieves 19.75% HTER and 87.46% AUC.
  • Adding DKG improves the result to 16.94% / 90.14%.
  • Further adding CSA advances the performance to 12.50% / 93.62%.
  • Finally, introducing AIAW boosts the performance to 8.86% / 97.14%.

In this cumulative ablation, HTER falls and AUC rises at each addition. The table does not test every separate combination of the three modules.

Next, let's look at the comparison of different Instance Whitening methods:

The experimental results show that methods like IW and GIW treat all covariance elements equally, which can inadvertently remove key discriminative features. While ISW shows some improvement, it still ignores the distributional differences between real and spoofed faces.

AIAW uses different selection fractions for live and spoof samples and constrains both branches. The best tested ratio in Table 5 is kr:ks=1:0.2k_r:k_s=1:0.2, with actual settings kr=0.3%k_r=0.3\% and ks=0.06%k_s=0.06\%. These are not 100% and 20% of covariance entries.

Visualization Analysis

Finally, to better understand how IADG aligns features, we look at the t-SNE projection.

In the t-SNE plots, the domain-aware method separates source live and spoof samples, but target live and spoof samples overlap more. With IADG, the target groups appear easier to separate. The figure does not draw a classifier decision boundary.

The more compact live features agree qualitatively with the design motivation.

Conclusion

For face anti-spoofing, drawing the boundary between real and fake is only the start; the harder challenge is handling variation.

These variations not only arise from the evolution of attack methods but also from the fragmentation of the data itself:

Different lighting conditions, devices, backgrounds, cultures, and shooting moments, each piece of data could be its own domain.

We once thought domain labels could control everything, but it turns out that more and more differences are hidden in those unclassifiable details. Rather than chasing uniform labels, it is better to extract the invariant signals from the style noise of each face.

From aligning the overall picture to understanding the individual; from classifying samples to dissecting style; perhaps we still don’t fully understand the essence of style, but IADG at least offers us a way to "coexist with style."