{"notes":[{"id":"face-antispoofing/rppg","category":"face-antispoofing","legacyPath":"face-antispoofing/1612-rppg","legacySlug":"1612-rppg","legacyPaths":[],"title":"rPPG","subtitle":"The Flicker of Life","author":"Z. Yuan","paperDate":"2016","publishedOn":"2025-05-13","paperTitle":"Generalized face anti-spoofing by detecting pulse from face videos","paperUrl":"https://doi.org/10.1109/icpr.2016.7900300","bibliography":{"title":"Generalized face anti-spoofing by detecting pulse from face videos","authors":["Xiaobai Li","Jukka Komulainen","Guoying Zhao","Pong-Chi Yuen","Matti Pietikainen"],"url":"https://doi.org/10.1109/icpr.2016.7900300","pdfUrl":null,"codeUrl":null,"preprint":null,"publication":{"venue":"ICPR","year":2016,"kind":"conference","url":"https://doi.org/10.1109/icpr.2016.7900300"},"citation":"@inproceedings{faceantispoofingrppg2016,\n  title = {{Generalized face anti-spoofing by detecting pulse from face videos}},\n  author = {Xiaobai Li and Jukka Komulainen and Guoying Zhao and Pong-Chi Yuen and Matti Pietikainen},\n  year = {2016},\n  booktitle = {ICPR},\n  url = {https://doi.org/10.1109/icpr.2016.7900300},\n  doi = {10.1109/icpr.2016.7900300}\n}\n","citationPath":"/content/papers/citations/face-antispoofing/rppg.e184784db406.bib"},"thumbnail":{"src":"/content/papers/assets/4c39640381020dd76587d643f18a91b30a8435306aa95d9f6042d8c05672d0fa.jpg","width":1808,"height":270},"locale":"en","path":"/content/papers/en/face-antispoofing/rppg.86da7615bfb0.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"face-detection-and-roi-tracking","text":"Face Detection and ROI Tracking","depth":3},{"id":"filtering-and-frequency-domain-transformation","text":"Filtering and Frequency Domain Transformation","depth":3},{"id":"feature-design","text":"Feature Design","depth":3},{"id":"classifier-design","text":"Classifier Design","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"3dmad-dataset","text":"3DMAD Dataset","depth":3},{"id":"real-f-dataset","text":"REAL-F Dataset","depth":3},{"id":"photo-and-video-attacks","text":"Photo and Video Attacks","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"face-antispoofing/cfrppg","relation":"next","topic":"Pulse signals for detecting face presentation attacks"}]},{"id":"face-antispoofing/cfrppg","category":"face-antispoofing","legacyPath":"face-antispoofing/1809-cfrppg","legacySlug":"1809-cfrppg","legacyPaths":[],"title":"CFrPPG","subtitle":"The Echo of a Heartbeat","author":"Z. Yuan","paperDate":"2018","publishedOn":"2025-05-11","paperTitle":"Remote Photoplethysmography Correspondence Feature for 3D Mask Face Presentation Attack Detection","paperUrl":"https://openaccess.thecvf.com/content_ECCV_2018/html/Siqi_Liu_Remote_Photoplethysmography_Correspondence_ECCV_2018_paper.html","bibliography":{"title":"Remote Photoplethysmography Correspondence Feature for 3D Mask Face Presentation Attack Detection","authors":["Si-Qi Liu","Xiangyuan Lan","Pong C. Yuen"],"url":"https://openaccess.thecvf.com/content_ECCV_2018/html/Siqi_Liu_Remote_Photoplethysmography_Correspondence_ECCV_2018_paper.html","pdfUrl":"https://openaccess.thecvf.com/content_ECCV_2018/papers/Siqi_Liu_Remote_Photoplethysmography_Correspondence_ECCV_2018_paper.pdf","codeUrl":null,"preprint":null,"publication":{"venue":"ECCV","year":2018,"kind":"conference","url":"https://openaccess.thecvf.com/content_ECCV_2018/html/Siqi_Liu_Remote_Photoplethysmography_Correspondence_ECCV_2018_paper.html"},"citation":"@inproceedings{faceantispoofingcfrppg2018,\n  title = {{Remote Photoplethysmography Correspondence Feature for 3D Mask Face Presentation Attack Detection}},\n  author = {Si-Qi Liu and Xiangyuan Lan and Pong C. Yuen},\n  year = {2018},\n  booktitle = {ECCV},\n  url = {https://openaccess.thecvf.com/content_ECCV_2018/html/Siqi_Liu_Remote_Photoplethysmography_Correspondence_ECCV_2018_paper.html}\n}\n","citationPath":"/content/papers/citations/face-antispoofing/cfrppg.a39b9479beac.bib"},"thumbnail":{"src":"/content/papers/assets/bd14addb06df9c23ece06dabad6f6d69606dadba3d799bc5295b83ec560b3e44.jpg","width":1224,"height":500},"locale":"en","path":"/content/papers/en/face-antispoofing/cfrppg.741053889028.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"learning-the-spectral-template","text":"Learning the Spectral Template","depth":3},{"id":"establishing-corresponding-features","text":"Establishing Corresponding Features","depth":3},{"id":"suppressing-background-noise","text":"Suppressing Background Noise","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"comparison-with-other-methods","text":"Comparison with Other Methods","depth":3},{"id":"cross-dataset-testing","text":"Cross-Dataset Testing","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"face-antispoofing/rppg","relation":"background","topic":"Pulse signals for detecting face presentation attacks"}]}],"text":"Although heart rate signals (rPPG) have been the subject of many studies in the past, most of them were focused on physiological monitoring applications.\n\nThis 2016 paper is an early in-depth study of pulse signals for face anti-spoofing.\n\nIt does not analyze textures or surfaces, but seeks a more fundamental proof of existence: **Is the heartbeat of this face truly happening?**\n\n## Defining the Problem\n\n\n\n![rPPG](/content/papers/assets/94d9386051ebc26a6597bc12a75e8c0b4c47338f786ff1494eea1e2005aac0eb.jpg)\n\n\n\n**Photoplethysmography (PPG)** measures changes in blood volume using light. Conventional PPG devices commonly contact the skin and use a light source and photodetector; camera-based, non-contact measurement is called remote PPG (rPPG). Both require sensors.\n\nWhen light shines on exposed skin (such as the earlobe, wrist, or cheek), local blood volume changes with each heartbeat, producing subtle variations in the absorption and reflection of light. These changes are imperceptible to the naked eye, but they leave faint clues in the intensity of reflected light. PPG captures these brightness variations and indirectly measures the presence of the heartbeat.\n\nBefore this paper was published, researchers had already demonstrated that **using a regular RGB camera, the same pulse signal (rPPG) can be restored from facial images**.\n\nEach heartbeat causes the face to undergo an extremely subtle change, even though this signal is undetectable by the human eye, appropriate Region of Interest (ROI) selection and filtering can help recover the signal in the frequency domain, although success is not guaranteed under every capture condition.\n\nFor years, FAS relied on \"visible differences\": texture flaws, color distortion, depth gaps, micro-expression lag, etc. These appearance-based strategies performed admirably against photo and video attacks. However, as the material and details of 3D masks became increasingly realistic, this line of defense started to crack:\n\n- **Appearance differences gradually disappear**: The maturation of silicone molds, 3D printing, and color coatings has made fake faces almost indistinguishable from real ones. Traditional LBP and texture models are no longer reliable.\n- **Variation in scenes and devices**: Different cameras and compression conditions distort the image frequency structure, making it difficult for models to generalize in real-world applications.\n- **Evolving attack methods**: As hardware barriers lower, new attack techniques continue to emerge, from screen playback to full-face masks, with each change potentially leading to a collapse of the defense.\n\nThis gradually brings up the question:\n\n> **When all the appearances of a face can be mimicked, can we still recognize \"life\" in the image?**\n\nThis paper focuses on temporal color variations in the images and instead asks a more fundamental question: **Does this face have a heartbeat?**\n\nBut this path is not easy.\n\nrPPG signals are weak and fragile, easily drowned out by environmental light variations, head movements, and encoding noise.\n\nFor the model to see it, it must first pass through layer upon layer of interference before finding that pulse belonging to a living person.\n\n## Solving the Problem\n\nExtracting the heartbeat from facial images may sound like something out of a science fiction novel, but it is indeed a real technology.\n\nIn this study, however, it is broken down into a well-defined signal processing pipeline.\n\nThe overall framework is divided into three simple parts: ROI extraction, signal transformation, and classification judgment.\n\n### Face Detection and ROI Tracking\n\n![model_arch](/content/papers/assets/4c39640381020dd76587d643f18a91b30a8435306aa95d9f6042d8c05672d0fa.jpg)\n\nFirst, the model's input is a video containing $n$ frames of a face.\n\nHere, the authors do not choose to use the entire face or just the eyes. Instead, they focus on the lower half of the face (cheeks, nose, mouth, chin), avoiding the forehead and eyes to reduce the risk of interference from hair and glasses.\n\nThe specific process is as follows:\n\n- Use the **Viola-Jones** detector to extract the face bounding box from the first frame;\n- Apply the **DRMF** model to annotate 66 facial landmarks;\n- Select 9 of these points to form the ROI, fixed in a stable bare skin area;\n- Use **KLT optical flow tracking** technology to dynamically update the ROI coordinates in each frame;\n- In each frame, compute the average RGB values of the pixels within the ROI, forming three raw time series signals: $r_{\\text{raw}}, g_{\\text{raw}}, b_{\\text{raw}}$.\n\nFor example, the raw signal of the red channel is:\n\n$$\nr_{\\text{raw}} = [r_1, r_2, \\dots, r_n]\n$$\n\nThese three signals are the uncleaned \"heartbeat samples,\" which may contain beats or be filled with noise.\n\n### Filtering and Frequency Domain Transformation\n\nTo extract the rhythm from the noise, the authors designed a three-phase signal cleaning process:\n\n- **Detrending**: Using _smoothness priors_ to remove slow variations and non-stationary trends;\n- **Moving Average**: Smoothing neighboring frame values to suppress spiky noise;\n- **Bandpass Filtering**: Using an FIR filter based on a Hamming window to preserve frequencies in the 0.7Hz to 4.0Hz range, which corresponds to a broad search range of 42 to 240 bpm, rather than a definition of normal resting heart rate.\n\nAfter filtering, the three signals are passed into the **Fast Fourier Transform (FFT)** to convert them into the frequency domain:\n\n\n\n![fft](/content/papers/assets/cd97e547d7b8a80d4faf8e6e3c0b898efd07574adc40d1298069bae288f3695b.jpg)\n\n\n\nAs shown in the figure above, the **real human face**'s Power Spectral Density (PSD) exhibits a dominant peak frequency (corresponding to the heartbeat), sometimes accompanied by a second harmonic peak. On the other hand, **attack samples** (masks/photos) often show scattered noise peaks, with low energy and no concentration.\n\n### Feature Design\n\nTo convert the PSD into classification input, the authors defined two features:\n\n- **$E$**: The maximum power value within the heartbeat frequency range (0.7–4.0 Hz);\n- **$\\Gamma$**: The ratio of the main peak energy to the total energy, used to measure the spectral concentration:\n\n$$\n\\Gamma = \\frac{E}{\\sum_{f \\in [0.7, 4]} e(f)}\n$$\n\nEach channel computes $E$ and $\\Gamma$, forming a 6-dimensional vector:\n\n$$\n\\left[ E_r, E_g, E_b, \\Gamma_r, \\Gamma_g, \\Gamma_b \\right]\n$$\n\n### Classifier Design\n\nFinally, a **linear Support Vector Machine (SVM)** classifies real faces and attacks. The authors keep its cost parameter at $C=1000$ in every experiment so the same setting can be used to compare the six-dimensional Pulse features across attacks.\n\n## Discussion\n\nHow far can a set of only six-dimensional physiological signal features go in different attack scenarios?\n\nThe authors chose to test three different types of attacks, corresponding to three public or self-constructed datasets:\n\n- **3DMAD**: The classic 3D mask attack dataset;\n- **REAL-F**: A new dataset with higher quality masks (self-constructed by the authors);\n- **MFSD**: A mobile device dataset that includes photo and video replay attacks.\n\n### 3DMAD Dataset\n\nIn the context of this study, 3DMAD was an early public dataset available for evaluating 3D mask attacks.\n\nIt includes 17 subjects and 255 video clips, divided into three sessions:\n\n- `s1` and `s2`: Real faces, with five clips recorded on different dates for each session;\n- `s3`: Attack samples, using 3D masks created by **ThatsMyFace**, worn by a third person to imitate the original subject.\n\nTo ensure sample independence, the experiment used a leave-one-subject-out design (LOOCV): each time, one subject is left out for testing, while the remaining 16 are split into training and development sets of eight subjects each, repeating this process 17 times.\n\nFeature and classification settings are as follows:\n\n- **Pulse features**: For each video, extract a 6-dimensional vector $[E_r, E_g, E_b, \\Gamma_r, \\Gamma_g, \\Gamma_b]$ from the first 10 seconds;\n- **Classifier**: A linear SVM is used with $C = 1000$, no parameter tuning;\n- **Baseline comparison**: Four classic LBP settings are used, including:\n\n  - `LBP-blk` (grayscale block histogram, 531 dimensions)\n  - `LBP-blk-color` (RGB block histogram, 1593 dimensions)\n  - `LBP-ms` (multi-scale grayscale histogram, 479 dimensions)\n  - `LBP-ms-color` (multi-scale RGB, 1437 dimensions)\n\nThe final evaluation metric used is **HTER (Half Total Error Rate)**, calculated as:\n\n$$\n\\text{HTER} = \\frac{\\text{FPR}(\\tau^*) + \\text{FNR}(\\tau^*)}{2}\n$$\n\nwhere $\\tau^*$ is the threshold selected at the development-set EER operating point.\n\nThe experimental results are shown in the table below:\n\n\n\n![3dmad](/content/papers/assets/4f85e6bfc6e399741c9b7bfd5f4bc05e47b9f7484761fba19530b6589287869d.jpg)\n\n\n\n**Pulse features achieve 7.94% test HTER on 3DMAD**, with 2.31% development EER and 4.71% test EER. This shows that six features and a linear classifier can distinguish some mask attacks in this dataset.\n\nHowever, all four **LBP baselines achieve a 0% error rate** on this dataset, seemingly a perfect victory. But behind this result lies an engineering reality:\n\n> **Training and testing use different subjects and their corresponding masks, but the mask fabrication process and capture conditions are similar, and visible printing textures remain.**\n\nThe zero-error LBP result therefore shows that it can pick up texture differences for these masks under similar capture conditions.\n\n### REAL-F Dataset\n\n\n\n![3dmad-realf](/content/papers/assets/0e0d2eee5607d2149041b95c6ccf7fa1190b35caf877b447014d06d558729cac.jpg)\n\n\n Left image shows the 3D mask from 3DMAD, right image shows the high-quality simulated mask from REAL-F\n\n\n\n\n\nIf 3DMAD is the training ground, REAL-F is much closer to the battlefield.\n\nThe REAL-F masks are finely crafted high-quality simulated skin, with materials, textures, and colors that more closely resemble skin than the masks used in 3DMAD. Because the authors did not have another suitable high-quality 3D mask dataset for this experiment, the authors purchased two REAL-F masks and recorded corresponding data to create a small test set:\n\n- **Dataset Composition**:\n\n  - 12 real samples (from 2 subjects);\n  - 12 attack samples (worn by a third person to mimic the original subject);\n  - All videos are 10 seconds long, with a resolution of 1280 × 760 and a frame rate of 30 FPS.\n\n- **Training Data**:\n\n  - Randomly select 8 subjects from 3DMAD as the training set, and 8 others as the development set;\n  - Use the entire REAL-F dataset as **unseen test data**, performing 100 random split tests.\n\n- **Evaluation Metrics**:\n\n  - Reported three key metrics: **EER** (Equal Error Rate), **HTER** (Half Total Error Rate), and FPR (False Positive Rate) at FNR = 0.1 and FNR = 0.01.\n\nThe experimental results are shown in the table below:\n\n\n\n![table2](/content/papers/assets/32b850b6bdc2f16787784ee54b8a186d9283c3d70d2582b498362426e94b9dce.jpg)\n\n\n\nWe can observe that all **LBP feature combinations decline sharply on REAL-F**, with much higher EER. Even `LBP-blk-color`, which has the lowest LBP EER of 20.42%, has a 48.67% attack FPR when FNR = 0.01.\n\nIn contrast, **Pulse features achieve 4.29% HTER and 1.58% EER** on these 24 unseen REAL-F videos.\n\nThis result highlights one key point:\n\n> **In this small cross-dataset experiment, pulse features adapted to unseen masks better than the compared LBP features.**\n\nLBP methods, which rely on textures, have two clear weaknesses:\n\n1. When faced with fake faces with overly realistic skin textures, they cannot extract distinguishing features;\n2. Once the training data is fixed, the model struggles to generalize to new types of masks.\n\nIn contrast, **pulse signals have a clear semantic definition (whether they exhibit real pulsations)**. They depend less directly on a particular mask texture. Detection still depends on signal visibility and on whether noise resembles a pulse.\n\nThe authors further analyzed the misclassification cases and visualized the $E_g$ distribution of all samples from 3DMAD and REAL-F as shown below:\n\n\n\n![eg-plot](/content/papers/assets/4ed44f2f7eb9fb8c3e301408f7461fd9d43939f4473042d85d393e00dda3cfee.jpg)\n\n\n Distribution of \\(E_g\\) in the Pulse features. Orange crosses represent mask attacks, mostly concentrated in the lower value range\n\n\n\n\n\nWe can observe the following:\n\n- Most attack samples (orange crosses) have very low $E_g$ values, corresponding to a noise pattern with no dominant frequency in the PSD;\n- Real samples generally have higher $E_g$ values, though with more variation, which is closely related to skin tone, face size, and lighting conditions;\n- The main misclassification cases are concentrated in one subject with darker skin and a smaller face area, conditions that weaken the rPPG signal or increase noise intensity.\n\nFrom here, we can conclude that the limitation of this method is that when the face is too dark or the image resolution is too low, the rPPG signal is drowned by environmental noise, leading to incorrect classification.\n\n### Photo and Video Attacks\n\nIn the real world, the easiest attack method to obtain is **a photo and a smartphone**.\n\nThis experiment uses the MSU MFSD dataset, which contains 35 subjects and a total of 280 videos, covering two typical types of attacks:\n\n- **Photo Attack** (print attack): Using high-quality printed photos;\n- **Video Attack** (video replay): Using an iPad or iPhone to replay a recorded face video;\n- Each subject includes 2 real samples, 2 photo attack samples, and 4 video attack samples;\n- The recording devices were a MacBook Air and Nexus 5 front camera, with resolutions of 640×480 and 720×480.\n\nTo evaluate the model's performance under different attack types, the authors divided the dataset into three groups:\n\n\n\n| Test Group | Real Samples | Attack Samples | Description         |\n| ---------- | ------------ | -------------- | ------------------- |\n| MSU-photo  | 70           | 70             | Photo attack        |\n| MSU-video  | 70           | 140            | Video replay attack |\n| MSU-all    | 70           | 210            | All attack samples  |\n\n\n\n- 15 subjects were used for training, and 20 were used for testing;\n- Pulse features were the same as in the previous section, with a 6-dimensional feature vector extracted from each video;\n- For comparison, only the most stable-performing `LBP-ms-color` was selected as the texture feature representative.\n\nThe experimental results are shown in the table below:\n\n\n\n![table3](/content/papers/assets/09741e5954127a2347dbfe354b66fbae58a4259acf754c4d57d8fd76067f378f.jpg)\n\n\nPulse and LBP features' EER performance under different attack scenarios\n\n\n\n\n\n- Pulse features achieve **5%** EER on photo attacks, compared with 10% for LBP;\n- For video attacks, EER rose to **35%**, indicating weak discrimination: sufficiently high-quality recordings can retain a genuine pulse signal;\n- LBP features achieve **5%** EER on video attacks, compared with 35% for Pulse, but 10% on photo attacks.\n\nThis confirms one limitation of the Pulse features:\n\n> **It can only distinguish \"whether there is a pulse,\" but cannot differentiate if the pulse is \"live\" or \"pre-recorded.\"**\n\nIf replay quality is sufficient, these six-dimensional Pulse features may fail to distinguish a live pulse from a recorded one.\nA static photograph has no physiological rhythm of its own, but periodic lighting or motion can still interfere with the signal.\n\nThe authors therefore cascade the Pulse model trained on MSU-photo with the color-LBP model trained on MSU-video. On MSU-all, the cascade reaches **7.5% EER**, compared with 36.67% for Pulse alone and 13.33% for LBP alone. These complementary cues help with the tested print and replay attacks; pulse detection alone does not cover both.\n\n## Conclusion\n\nThis paper did more than propose a new method: it opened another branch in the FAS toolkit.\n\nAt the time, rPPG was studied mostly for physiological monitoring. Here the authors gave it a role in anti-spoofing: instead of relying on how a face looks, ask whether a measurable rhythm is present. That was a bold change of viewpoint when texture, lighting and depth dominated the discussion.\n\nThe faint pulse also proved useful. In the small cross-dataset test with two REAL-F masks, six-dimensional Pulse features outperformed the much larger texture descriptors. That comparison supports this experiment's finding.\n\nBut the method has obvious weak spots:\n\n- It struggles with video replay, because a recorded face can retain the pulse signal.\n- Lighting, image quality, skin tone and motion can make that signal difficult to recover.\n\nMost of all, the pulse’s capture provenance still needs verification. Better signal extraction or a larger model may reduce some errors, but they cannot by themselves supply that missing provenance.\n\n> **Sometimes, changing the cue we observe reveals more than simply enlarging the model.**\n\nLater work developed variants such as CFrPPG, MCCFrPPG and MSTCT, improving signal stability or combining physiological and appearance cues. FAS methods have changed repeatedly, yet physiological signals remain a useful line of evidence against convincing visual spoofs. Even when a system uses other methods today, this paper's change of perspective is worth keeping in mind.\n","imageSizes":{"/content/papers/assets/94d9386051ebc26a6597bc12a75e8c0b4c47338f786ff1494eea1e2005aac0eb.jpg":{"width":1072,"height":484},"/content/papers/assets/4c39640381020dd76587d643f18a91b30a8435306aa95d9f6042d8c05672d0fa.jpg":{"width":1808,"height":270},"/content/papers/assets/cd97e547d7b8a80d4faf8e6e3c0b898efd07574adc40d1298069bae288f3695b.jpg":{"width":1116,"height":384},"/content/papers/assets/4f85e6bfc6e399741c9b7bfd5f4bc05e47b9f7484761fba19530b6589287869d.jpg":{"width":800,"height":408},"/content/papers/assets/0e0d2eee5607d2149041b95c6ccf7fa1190b35caf877b447014d06d558729cac.jpg":{"width":624,"height":600},"/content/papers/assets/32b850b6bdc2f16787784ee54b8a186d9283c3d70d2582b498362426e94b9dce.jpg":{"width":1076,"height":420},"/content/papers/assets/4ed44f2f7eb9fb8c3e301408f7461fd9d43939f4473042d85d393e00dda3cfee.jpg":{"width":1224,"height":400},"/content/papers/assets/09741e5954127a2347dbfe354b66fbae58a4259acf754c4d57d8fd76067f378f.jpg":{"width":928,"height":328}}}
