Back to paper library
ECCV 2018Face anti-spoofingPaper notes

CFrPPG

The Echo of a Heartbeat

Original paper

Remote Photoplethysmography Correspondence Feature for 3D Mask Face Presentation Attack Detection

Paper authors

Si-Qi Liu · Xiangyuan Lan · Pong C. Yuen

Publication
ECCV 2018Conference
Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@inproceedings{faceantispoofingcfrppg2018,
  title = {{Remote Photoplethysmography Correspondence Feature for 3D Mask Face Presentation Attack Detection}},
  author = {Si-Qi Liu and Xiangyuan Lan and Pong C. Yuen},
  year = {2018},
  booktitle = {ECCV},
  url = {https://openaccess.thecvf.com/content_ECCV_2018/html/Siqi_Liu_Remote_Photoplethysmography_Correspondence_ECCV_2018_paper.html}
}
On this page

rPPG, essentially evolved from PPG (Photoplethysmography), is a non-contact heartbeat measurement technique.

It does not require the attachment of sensors; with just an RGB camera, it can observe the color changes in the skin caused by heartbeats under natural light, thereby recovering physiological signals.

In the FAS task, this mechanism offers a way to bypass the visual appearance of the image and directly observe signs of life.

In theory, as long as stable rPPG signals can be extracted from the image, it can effectively distinguish between real faces and 3D masks, and may help detect photographs that have no pulse of their own. High-quality video replay can preserve a pulse, however, so capture provenance also needs verification.

Defining the Problem

In real-world scenarios, rPPG signals are often not so pure.

The image below shows three common rPPG signal patterns, shown in the original figure as a genuine face, a mask, and an ambiguous signal affected by interference, from left to right.

  • First (left): Ideal scenario. The rPPG signal from a real face shows a distinct main frequency in the frequency domain, representing the heartbeat rhythm, with background noise relatively weak. This is the kind of liveness evidence we expect the model to capture.

  • Second (middle): An ideal mask sample. Its spectrum lacks a clear, concentrated pulse peak, unlike the genuine face on the left.

  • Third (right): An ambiguous signal. Strong peaks may come from a pulse or from motion and lighting interference. A genuine face can lose its clear peak under poor capture conditions, while periodic noise can create peaks for a mask. The maximum alone is insufficient.

These three patterns highlight a harsh reality:

rPPG signals are fragile, and the maximum spectral intensity is not sufficient as the sole liveness indicator.

Traditional spectral peak methods focus the model's attention on the strongest signal but overlook the overall structure and consistency of the spectrum. The heartbeat signal should ideally remain steadily hidden within the skin's subtle color changes, waiting to be extracted, amplified, and analyzed.

However, the problem lies in: Blood-flow-related color variations are just one of the factors; more often, noise is the dominant theme.

A core assumption of rPPG methods is: "The main frequency of the heartbeat will form a clear peak in the spectrum."

Some earlier PAD methods compared in this paper use spectral peaks as liveness features. CHROM and POS are rPPG signal-extraction methods; they should not be equated with a particular PAD decision rule.

However, once environmental factors like camera shake, brightness flicker, or facial displacement appear, even masks can exhibit significant periodic signals. Under low resolution or low-light conditions, the real facial signals may be obscured by environmental noise, causing the spectral peak to disappear.

Traditional cross-correlation operations can enhance shared frequencies but cannot distinguish whether "the shared frequency is the heartbeat or synchronized background noise."

Thus, we must ask:

If noise and signal resonate at the same frequency, which one should we trust?

Solving the Problem

Existing rPPG methods typically treat the "spectral peak" as an indicator of heartbeat intensity.

While this works under ideal conditions, as pointed out in previous analyses, once noise and real signals resonate at the same frequency, relying solely on intensity can lead to misclassification. CFrPPG (rPPG Correspondence Feature) is a new method proposed to address this fragile assumption.

Learning the Spectral Template

The core concept of CFrPPG is:

  • If a face is real, then its different regions should exhibit consistent heartbeat signals at the same frequency.

To capture this "consistency between regions," the authors design a learnable spectral template ww, which is not directly derived from a single signal but is instead inferred from the rPPG spectra of multiple regions of the face.

For template learning, we extract rPPG signals from several larger facial regions, apply cross-correlation preprocessing, and obtain their spectra s1,s2,…,sN{s_1, s_2, \dots, s_N}. Later, smaller overlapping regions supply the final correspondence features.

These spectra represent the energy distribution across different frequencies for each region but are mixed with noise and variability. Therefore, we want to learn a template ww that enhances the "consistent heartbeat frequency" while ignoring the diversity introduced by noise.

This goal can be achieved with the following equation:

min⁡w∑i=1N∥Siw−y∥22+λ∥w∥22\min_w \sum_{i=1}^N \| S_i w - y \|_2^2 + \lambda \| w \|_2^2

Here, SiS_i is the circulant matrix formed by sis_i, and yy is a fixed one-dimensional Gaussian target with a peak at the center of the correlation response. It is not a known heart-rate label: regression produces a matching response over shifts of the spectral vector.

This design allows us to integrate the "consistent frequency components across all regions" into a single filter ww.

To speed up the computation, the authors use the property of circulant matrices being diagonalizable in the frequency domain and transform the optimization problem into an element-wise operation in the frequency domain:

w^=∑i=1Ns^i∗⊙y^∑i=1Ns^i∗⊙s^i+λ\hat{w} = \frac{\sum_{i=1}^N \hat{s}_i^{*} \odot \hat{y}}{\sum_{i=1}^N \hat{s}_i^{*} \odot \hat{s}_i + \lambda}

Here, s^i\hat{s}_i is the DFT-processed form of the ii-th region's spectrum, ∗^* denotes element-wise complex conjugation, and ⊙\odot represents element-wise multiplication. This means we no longer handle large matrix multiplications but instead use simple addition and division in the frequency domain to learn the filter.

This is like drawing a common spectral shape from the local signals in one face video and then checking each region's correspondence with that template.

Establishing Corresponding Features

Once the spectral template ww is learned, the next task is to check:

  • Does the rPPG signal in each region of the face "respond" to this template?

We apply another DFT to each spectral vector sis_i, obtaining s^i\hat{s}_i, and multiply it element-wise by w^\hat{w}. This DFT accelerates circular convolution over the spectral vector; it is distinct from the earlier transformation of temporal signals into pulse spectra:

r^i=s^i⊙w^\hat{r}_i = \hat{s}_i \odot \hat{w}

An inverse DFT then gives the real matching response ri=IDFT⁡(r^i)r_i=\operatorname{IDFT}(\hat{r}_i), whose peaks can be analyzed. It measures spectral-template correspondence; its shift coordinate is not itself a heart rate in hertz.

To quantify this resonance, the authors adopt the Peak-to-Sidelobe Ratio (PSR), a common sharpness indicator for filter responses:

PSRi=peakri−μriσri\text{PSR}_i = \frac{\text{peak}_{r_i} - \mu_{r_i}}{\sigma_{r_i}}

Where:

  • peakri\text{peak}_{r_i} is the maximum value of the response sequence
  • μri\mu_{r_i} and σri\sigma_{r_i} are the mean and standard deviation of the non-peak regions

The paper's printed PSR expression multiplies by the standard deviation. The equation above follows the standard peak-to-sidelobe ratio definition and uses separate notation for the response sequence and its PSR value. The author's implementation has not been checked to resolve that discrepancy.

In other words, the higher the PSR, the sharper and more focused the response, which is more likely to represent a real heartbeat signal; while noise or incoherent signals will produce flat, blurry, or multi-peak responses, resulting in a lower PSR.

Finally, we concatenate the PSR values from all regions into a multi-dimensional liveness feature vector:

x=[PSR1,PSR2,…,PSRN]x = [\text{PSR}_1, \text{PSR}_2, \dots, \text{PSR}_N]

This vector represents the final CFrPPG feature, which no longer relies on a single frequency intensity but instead assesses the correspondence between all regions of the face and the template to determine whether the face has a consistent, real heartbeat rhythm.

This design, which starts from "overall consistency," allows the model to better resist interference from local noise and erroneous signals.

Suppressing Background Noise

In real-world applications, rPPG signals are easily contaminated by global interference, such as slight camera shake, vibrations from handheld shooting, or flickering indoor lighting. These interferences generally have two characteristics: synchronous across the entire face and stable periodicity, making them hard to distinguish from heartbeat signals in the frequency domain, and easily causing the model to produce incorrect responses.

This means that even a mask without any physiological signals can, as long as the background noise is stable enough, generate spurious heartbeat traces in the frequency domain.

To solve this problem, CFrPPG introduces background region rPPG spectra as negative samples during the learning of the spectral template. These regions are located outside the face, and rPPG signals are extracted using the same method as for the face. However, these regions should not exhibit any physiological rhythm from blood flow.

These background spectra are considered examples of "should not produce a response," and are incorporated into the template training objective function in the form of a penalty term:

min⁡w∑i=1M∥Sitw−y∥22+λ∥w∥22+γ∑k=1K∥Sknw∥22\min_w \sum_{i=1}^M \| S_i^t w - y \|^2_2 + \lambda \|w\|^2_2 + \gamma \sum_{k=1}^K \| S_k^n w \|^2_2

Where:

  • SitS_i^t: rPPG spectra from facial regions, used to capture heartbeat consistency
  • SknS_k^n: spectra from the background, used to suppress global interference
  • yy: a fixed one-dimensional Gaussian target, encouraging a sharp response when spectra align
  • γ\gamma: a coefficient controlling the penalty strength, balancing heartbeat learning and background suppression

In this way, the objective encourages the learned template ww to do two things:

  • It produces strong and focused responses at the real heartbeat frequency
  • It maintains a very low response to background common periodic noise

The design means we don't directly "exclude" background frequencies but instead let the model learn to ignore those periodic patterns that always appear in the wrong places during training.

In other words, we don't just ask the model "where do you see the heartbeat," but also "where shouldn't you see the heartbeat."

Under the interference conditions tested in the paper, this reduces the effect of synchronized background signals on the correspondence features. The implementation learns its template from three larger face regions and four background regions, then extracts features from nine smaller overlapping face regions and classifies them with a linear SVM.

Discussion

Comparison with Other Methods

As shown in the table above, under the 3DMAD dataset, all methods exhibit relatively high overall performance, reflecting that this dataset was captured under controlled lighting conditions, and the Thatsmyface mask used still has noticeable differences in material compared to a real face. Appearance-based methods (especially MS-LBP) therefore achieve the best results in this scenario.

Compared with the rPPG methods LrPPG and GrPPG, CFrPPG has slightly better AUC and HTER on 3DMAD. Its test HTER is 6.82%, versus 8.57% for LrPPG. The CNN baseline, however, reaches 6.07% test HTER, so CFrPPG does not lead every method here; MS-LBP remains best on this dataset.

In addition, this experiment uses a random subject index for leave-one-out cross-validation (LOOCV), conducted over 20 rounds with 17 tests per round for complete cross-validation to reduce the randomness of data splitting. CFrPPG consistently maintains stable performance across multiple evaluations, showing that its architecture possesses consistency across samples.

Cross-Dataset Testing

The authors further designed cross-dataset experiments to evaluate the model's generalization ability in the face of unknown environments.

Specifically, the experiment is divided into two directions:

  • 3DMAD → HKBU-MARsV2: Training on a clean, controlled dataset, then testing on a dataset with different indoor lighting and mask types
  • HKBU-MARsV2 → 3DMAD: Training on the dataset with greater lighting and mask variation, testing in a standard environment

Each setup is executed with 20 rounds of random group testing to avoid bias in subject allocation.

In Table 4, CFrPPG has the lowest HTER in both directions: 2.51% for 3DMAD → HKBU-MARsV2 and 2.55% for the reverse direction. Its lead over the compared methods is larger than on 3DMAD alone:

CFrPPG not only stabilizes recognition in familiar environments but also maintains recognition ability under cross-dataset transitions.

The authors attribute this to correspondence between regional spectra and a learned template, rather than reliance on a single spectral maximum. The cross-dataset results support better discrimination than the methods in this table, without proving that the template alone caused the difference.

In contrast, the appearance-based methods here (including MS-LBP and CNN) decline sharply in these cross-dataset tests, suggesting sensitivity to the visual conditions and mask types on which they were trained.

Conclusion

rPPG once carried the hope that we could read evidence of life from images without touching the person. Researchers tried to recover a blood-flow rhythm from tiny changes in light. Then engineering reality answered:

Lighting is hard to control, noise is hard to remove, and frequency features are fragile.

rPPG is a difficult kind of engineering romance.

CFrPPG tries to make that pursuit steadier. It uses template matching, regional spectral correspondence and background-noise modeling to handle uncertain capture conditions. Instead of trusting a single high peak, it asks whether multiple facial regions agree, then asks whether the same pattern also appears where no heartbeat should be. In the tested settings, that shared evidence is more useful than one extreme value.

From the FAS perspective, rPPG may look like a branch that rose quickly and then became less prominent. But this paper does not mark its end: work on physiological cues continues, and its evaluation covers two mask datasets. Even as methods change, the question CFrPPG asks still lingers: when we hear a rhythm in an image, where did it come from?