On the Effectiveness of Local Binary Patterns in Face Anti-spoofing
- Publication
- BIOSIG 2012Conference
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@inproceedings{faceantispoofinglbp2012,
title = {{On the Effectiveness of Local Binary Patterns in Face Anti-spoofing}},
author = {Ivana Chingovska and André Anjos and Sébastien Marcel},
year = {2012},
booktitle = {BIOSIG},
url = {https://publications.idiap.ch/publications/showlist/year/44}
}
Here, we can define a few important terms:
- Active Liveness: The user performs actions or commands such as blinking or turning their head, which are actively requested by the system. The system checks the response for evidence of a live presentation. Establishing whose face it is remains a separate recognition task.
- Passive Liveness: The system requires no specific challenge gesture and analyzes texture, image-formation or natural-motion cues during ordinary capture. Motion is not always required.
In simple terms, passive liveness assesses presentation authenticity from normal capture, without asking the user to perform challenge gestures. This approach enhances user convenience and reduces the burden of the identity verification process.
It sounds good to reduce the burden, but upon closer inspection, "active liveness" based only on fixed, predictable gestures may also be exploited by attackers. For example, an attacker could record the user’s actions and then play these recordings to the verification system, bypassing identity verification.
Defining the Problem
The authors argue that past literature, when constructing attack scenarios and datasets, often made simplified assumptions that fail to fully reflect the diversity and complexity found in real-world applications. Therefore, the author attempts to propose more practical improvements.
In this paper, several important attack methods are defined:
- Print Attack: The attacker uses a printed photo to deceive the system.
- Mobile Attack: The attacker uses a mobile screen to display photos or videos.
- Highdef Attack: The attacker uses a high-resolution screen to display photos or videos.
In the figure below, the first column represents a real user, the second column shows a Print Attack, the third column is a Mobile Attack, and the fourth column is a Highdef Attack.
To better simulate these attack scenarios, the authors created a new public dataset, REPLAY-ATTACK, and reconsidered feature extraction design using the multi-scale LBP method proposed in the literature:
- A simplified variant of Local Binary Patterns (LBP) is adopted, focusing on the key information in image textures to improve detection efficiency.
This approach is primarily inspired by the following research:
For easier comparison, the authors refer to this paper as [7], and it is contrasted with the current paper.
The authors argue that compared to literature [7], specific improvements have been made in the following areas:
- Attack scenarios and data more closely resemble real-world applications: Literature [7] only uses the NUAA dataset and includes only printed photo attacks with static images. This is still too simplistic for simulating real-world attacks.
- Simplified feature design, enhancing efficiency and practicality: Although the multi-scale LBP features in literature [7] perform well in classification, their 833-dimensional feature vector introduces considerable computational burden.
Solving the Problem
The above diagram shows the entire anti-spoofing algorithm workflow based on LBP.
Starting from the original video, the method detects a face in each frame, normalizes the face region to 64×64 pixels, and extracts features from that frame. If detection fails, it reuses the last successfully detected face position; if no previous position exists, it discards the frame.
Next, feature extraction employs two different strategies:
- Option 1: Calculate the LBP features for the entire image, generating a 59-dimensional histogram. This histogram becomes the texture feature vector of the image.
- Option 2: Divide the image into 3×3 blocks, compute LBP features for each block independently, and then concatenate the histograms of all blocks, forming a 531-dimensional feature vector.
Finally, these feature vectors are input into a classifier for binary classification, determining whether the input is genuine or a spoof.
Next, we will look at the details of each step.
Local Binary Patterns
For any pixel in a grayscale image , the LBP operator is based on a 3×3 neighborhood.
Let the central pixel value be , and its surrounding 8 neighboring pixels be .
For each neighboring pixel, we define the comparison function:
Therefore, the LBP value for the central pixel is calculated as:
This formula converts the 8 comparison results (each being 0 or 1) into an 8-bit binary number, with a corresponding value range from to .
Uniform LBP
In the paper, the Uniform LBP is referred to as .
Among the many possible 8-bit binary patterns, some are considered "uniform," meaning that in a circular check, the number of transitions from 0 to 1 or from 1 to 0 does not exceed 2.
Formally, for a binary sequence (where ), if the number of transitions satisfies
then the pattern is called uniform.
Non-uniform patterns are typically merged into a single category. This significantly reduces the dimensionality of the histogram. For a 3×3 neighborhood, there are 58 uniform patterns plus one bin for all remaining patterns, giving a 59-dimensional feature vector.
Feature Vector Generation
-
(1) Per-Image Features
- Full Image Histogram: The LBP value is computed for each pixel in the entire face region (typically the face bounding box is normalized to 64×64 pixels), and the occurrences of each uniform pattern are counted to form a 59-dimensional histogram. After normalization, this histogram serves as the texture feature vector for the image.
-
(2) Per-Block Features
- Block Histogram: The normalized face image is divided into 3×3 blocks, and a 59-dimensional LBP histogram is calculated for each block. The histograms of all 9 blocks are then concatenated to form a 531-dimensional feature vector. This approach allows finer capture of texture differences in local areas of the image, such as the forehead, cheeks, etc.
Extended Variants of LBP
In addition to the basic LBP, the paper also experiments with other variants, mainly including:
- tLBP (Transitional LBP): This method cyclically compares two neighboring pixels adjacent to the central pixel to form a binary pattern. This method emphasizes the transitional characteristics between adjacent pixels.
- dLBP (Direction-coded LBP): dLBP encodes intensity changes along the four basic directions passing through the central pixel, with each direction represented by two bits, thus capturing more directional information.
- mLBP (Modified LBP): mLBP is conceptually similar to the Modified Census Transform (MCT), where instead of directly comparing the central pixel to the neighboring pixels, the neighboring pixels are compared to the average value of the 3×3 region, and the comparison result with the central pixel is discarded. This reduces the noise impact that the central pixel might introduce.
Classifier
Since the feature vector obtained from the LBP operator is essentially a normalized histogram, the initial classifier typically uses the histogram comparison method. The specific process is:
- During training, histograms for all genuine access images are computed and averaged to obtain a reference histogram.
- For a test image, its LBP histogram is computed and the distance is used to measure the similarity between this histogram and the reference histogram, producing a classification score.
The paper also tests Linear Discriminant Analysis and Support Vector Machines. On REPLAY-ATTACK, both achieve lower test HTER than basic histogram comparison.
Dataset Overview
This paper introduces a public dataset named REPLAY-ATTACK, specifically designed for studying anti-spoofing techniques in face recognition systems. Below are the main configurations of the dataset:
- Dataset Name: REPLAY-ATTACK
- Dataset Size: Contains 50 participants, with a total of 1,200 video clips (24 video clips per participant).
- Video Length and Format: Real-access clips are 15 seconds long (375 frames); attack clips are about 10 seconds long. They are recorded at 320×240 (QVGA) and 25 frames per second.
- Recording Scenarios: Videos are recorded in two static environments:
- Controlled Conditions: Background is a uniform scene with fluorescent lighting.
- Adverse Conditions: Background is non-uniform, and lighting comes from natural daylight.
- Recording Equipment: Videos are recorded using an Apple 13-inch MacBook laptop, with participants sitting in front of the camera.
- Attack Methods: The dataset includes three common spoofing attack techniques:
- Print Attack: High-resolution photos are printed on A4 paper using a laser printer (Triumph-Adler DCC 2520).
- Mobile Attack: Photos or videos are captured using an iPhone 3GS and played on the device’s screen.
- Highdef Attack: High-resolution content is recorded with a Canon PowerShot SX150 IS and played on an iPad screen (1024×768).
- Attack Presentation Modes: Each attack type has two implementation modes:
- Hand-based: The operator holds the device to perform the attack, which causes slight shaking and sometimes misleads the blink detector.
- Fixed-support: The device is placed on a fixed stand, without shaking.
- Video Classification: The dataset is randomly divided into three subsets: training, development, and testing sets, with no overlap between identities. Attack videos are also categorized by attack method (hand-held/fixed).
- Training Recommendation: The training set fits the model; the development set supports parameter tuning, stopping decisions and threshold selection. The test set is reserved for final evaluation.
- Additional Data: 100 "enrollment videos" recorded under the same lighting conditions are provided for training basic face recognition models to assess attack quality.
- Dataset Accessibility: The authors publish the dataset, source code and evaluation protocol. Downloads and use remain subject to each resource's original license terms.
- Dataset URL: REPLAY-ATTACK
The table above shows the distribution of video clips across different subsets in the REPLAY-ATTACK dataset.
Evaluation Method
The detection system may produce two types of errors:
- False Rejection Rate (FRR): The proportion of genuine access attempts wrongly rejected (legitimate access misclassified as an attack).
- False Acceptance Rate (FAR): The proportion of attack samples wrongly accepted (attacker misclassified as legitimate).
The combined metric for these two errors is called the Half Total Error Rate (HTER), calculated as:
Since FAR and FRR are both affected by the threshold, the ROC curve is commonly used to show performance at different thresholds, and the EER (Equal Error Rate), the point where FAR equals FRR, is used to select the threshold on the development set. The HTER is then reported on the test set.
- A lower HTER score indicates that the classifier is more accurate at distinguishing between genuine and spoofed face access.
In the experiments, the authors used three datasets for evaluation: REPLAY-ATTACK, NUAA, and CASIA-FASD.
Since the NUAA and CASIA-FASD datasets lack independent development sets, 5-fold cross-validation was used, and the average HTER was reported.
Discussion
Comparison of Different LBP Features
The table above shows the HTER results on the REPLAY-ATTACK dataset when different LBP feature variants are evaluated using histogram comparison as the classifier.
- LBP: This is the commonly used "uniform LBP," where LBP values are calculated using a 3×3 neighborhood, and only patterns with no more than two transitions are kept, generating a 59-dimensional histogram.
- mLBP (modified LBP): In this calculation, in addition to performing binary comparisons, the average pixel intensity of the 3×3 neighborhood is calculated and compared to each neighboring pixel (the comparison between the central pixel and the average is discarded), resulting in an improved texture description.
The table also evaluates tLBP and dLBP. Among all four features, mLBP has the lowest test HTER at 33.68%, followed by standard LBP at 34.01%, tLBP at 35.35%, and dLBP at 40.26%.
Although mLBP has a slightly lower test HTER, the authors use the more common LBP in subsequent comparisons to keep the methods comparable. mLBP also requires an extra neighborhood-mean calculation.
Comparison of Different Datasets
- ∗ (asterisk): Indicates that the results in this row were reproduced using the Bob toolbox for the methods described in [7], including feature extraction and classification.
- ‡ (double dagger): Indicates the results reported in the original paper [7], using Matlab for LBP feature extraction and LibSVM for classification.
- § (section sign): Indicates that the features were provided by the original authors of [7] and classified using the Bob toolbox.
To compare the performance of different methods across various datasets, the authors conducted experiments on three public datasets and used LBP features to compute the HTER for each as the metric. REPLAY-ATTACK uses its fixed training/development/test split. For NUAA and CASIA-FASD, the training data are divided into five folds, each serving in turn as the development set; the reported test HTER is averaged over those five runs.
Regarding feature extraction strategies, REPLAY-ATTACK uses full-image features, while NUAA and CASIA-FASD use block-based features.
The results show that, depending on the dataset, the choice of features significantly influences classification performance:
- On the REPLAY-ATTACK dataset, full-image and nine-block features have similar HTER (34.01% versus 34.30% with comparison). The authors choose the full-image feature for its lower computational and memory costs.
- On the other hand, for the NUAA and CASIA-FASD datasets, block-based features performed better.
Additionally, the NUAA results show very low development HTER but markedly higher test HTER. Because the development folds come from the training data, the authors regard this as a possible sign of overfitting and call for an independent development set.
From the model design perspective, on the REPLAY-ATTACK test set, the 833-dimensional feature from [7] with SVM achieves a test HTER 1.29 percentage points lower than this paper's 59-dimensional feature with SVM (13.87% versus 15.16%). The larger feature vector also increases the SVM's size. Even the 59-dimensional feature with SVM uses 25,717 support vectors, so test-time computation needs to be weighed against performance.
Conclusion
In an era before deep learning became mainstream, the authors clearly pointed out that even if face recognition models perform well, without effective anti-spoofing detection, the system's credibility cannot be truly guaranteed. This viewpoint still holds today, and "Face Anti-Spoofing" has become an indispensable core module in biometric recognition systems.
The REPLAY-ATTACK dataset was one of the few public resources at the time focused on video-based attacks and covering various types of spoofing. Not only does it include the three attack types: Print, Mobile, and Highdef, but it also considers different devices and recording conditions, with a complete training and testing protocol. This dataset has established a baseline for subsequent FAS research and contributed to the trend of dataset standardization.
Today, LBP features cannot match the performance of CNN and Transformer-based models. Still, their computational efficiency and reproducibility can be useful in resource-constrained settings, and this work helped set a direction for the field. Whether the method suits a particular deployment still depends on the devices and attack conditions being tested.