Face Liveness Detection from A Single Image with Sparse Low Rank Bilinear Discriminative Model
- Publication
- ECCV 2010Conference
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@inproceedings{faceantispoofingslrbd2010,
title = {{Face Liveness Detection from A Single Image with Sparse Low Rank Bilinear Discriminative Model}},
author = {Xiaoyang Tan and Yi Li and Jun Liu and Lin Jiang},
year = {2010},
booktitle = {ECCV},
url = {https://doi.org/10.1007/978-3-642-15567-3_37},
doi = {10.1007/978-3-642-15567-3_37}
}
This time, let's take a closer look at how liveness detection developed.
We'll start with a few papers from the early days of this field.
Defining the Problem
Liveness detection has been the subject of numerous studies since before 2010. The methods of that time relied on facial movements (such as blinking, small facial changes), skin texture, depth information, and other features to recognize a live person.
However, a detector that relies on a few predictable cues may invite attacks tailored to those cues. For example, detecting facial movements can be fooled by playing a short video, shaking a photo, or using a dual-photo simulation.
What if we introduce a 3D camera or thermal imaging device?
These sensors can add useful evidence, but at the time of this paper they also imposed extra hardware costs and deployment constraints. We normally carry a phone; requiring a separate thermal camera for occasional verification is a different proposition.
Therefore, the authors of this paper propose to distinguish a live face from a photo using "a single image" from an ordinary camera.
For example, as shown in the figure below, which images are photos of faces (reproduced), and which ones are real human faces?
The answer is: The first column from the right is real, the rest are reproduced photos.
What! You say you can't tell with your naked eye?
Exactly, and that’s the problem.
Solving the Problem
The authors first examined the data with Principal Component Analysis (PCA) and found that the live and spoof image distributions overlapped substantially, making the two difficult to separate directly.
To solve this issue, the authors adopted the "Lambertian Reflection Model" as the basis for analysis, aiming to find distinguishing factors from the perspective of how light interacts with surfaces. They approached the problem as a "binary classification" issue—simply put, classifying the image into "live" or "spoof" categories.
The Lambertian Reflection Model is a mathematical model that describes how a surface scatters incident light, assuming the surface is an ideal diffuse reflector. Under this idealization, reflected radiance does not depend on the viewing direction.
This property comes from Lambert’s cosine law, and its basic formula is:
Where:
- : Brightness at position in the image;
- : Camera's light response factor, typically treated as a constant;
- : Reflection coefficient, representing the surface's diffuse reflection characteristics;
- : Light source intensity;
- : Cosine of the angle between the light ray and the surface normal vector, written as .
According to this model, the smaller the angle between the light and the surface normal (i.e., approaches 1), the higher the reflection intensity. Conversely, the larger the angle (i.e., approaches 0), the lower the reflection intensity. This provides an approximate model for skin and paper; actual materials can also exhibit specular reflections and other unmodeled effects.
The authors further applied this model to the liveness detection scenario, considering two images obtained under the same lighting conditions:
- Live image
- Spoof image
According to the Lambertian model, both can be represented as:
Assuming is constant, we have:
Further breaking it down:
Where , decomposing the image into reflection components and lighting components , highlighting the potential for distinguishing between live and spoof images.
To estimate the lighting component , the authors used the Logarithmic Total Variation (LTV) smoothing method:
Where is set to 0.5.
Next, the authors computed the reflection component using Land’s Retinex formula:
This decomposition allows us to observe that the of live images contains more surface normal information, while the of spoof images is relatively smooth, indicating less detail in lighting and structure. This is because spoof images, after being captured by a camera and printed, lose quality and details.
To strengthen these differences, the authors also used a Difference of Gaussian (DoG) filter to extract high- and mid-frequency variations in the image:
- Internal Gaussian standard deviation is 0.5
- External Gaussian standard deviation is 1.0
After applying DoG, the high-frequency components of the live image (especially in the horizontal direction) become more prominent, while the mid-frequency components of the spoof image appear distorted and blurred.
Additionally, under the same lighting conditions, the ratio of lighting components is mainly influenced by the surface normals and , which means that the "reference image of the real user" can help identify spoof attacks.
For different lighting conditions, the authors proposed the following strategy:
- Use Singular Value Decomposition (SVD) to learn common lighting directions from the training data, forming a lighting matrix ;
- Reconstruct a lighting image with the estimated surface-normal matrix and , solve for coefficients , and use those coefficients as classifier input features.
However, LTV decomposition can let some texture detail leak into , making its distribution nonlinear. The experiments therefore do not use those reconstructed lighting coefficients; the authors learn classifiers directly from the latent samples obtained by decomposition or filtering.
For the classifier design:
- Sparse Logistic Regression: Adds regularization to standard logistic regression to control overfitting in high-dimensional samples.
- Sparse Low-Rank Bilinear Logistic Regression: Directly operates on the 2D image structure, learning a low-rank projection matrix , applying regularization on and to zero out some rows, achieving sparsity and low-rank properties. It is optimized using alternating coordinate descent until convergence.
A separate extension maps samples into a feature space using a kernel mapping , then applies sparse logistic regression to obtain a nonlinear model. This is an alternative to the low-rank bilinear extension.
The authors treat the nonlinear model after kernel mapping as a non-parametric probabilistic model: its feature dimension grows with the number of training samples, while regularization encourages a sparse solution. The paper compares its solver's computational complexity with pSVM, RVM and IVM.
After all the formulas, the authors also kindly provide an example image to illustrate the entire model architecture.
The image above shows the latent samples derived from "real user images" (top row) and "spoof images" (bottom row). Through different image processing and spectral analysis techniques, key features are extracted to distinguish between the two.
Let’s explain the meaning of each column:
- Column 1: This is the original image, with the top row showing the real user image and the bottom row showing the spoof image (reproduced photo).
- Column 2: This is the lighting image (µ Image) estimated by LTV, showing the lighting distribution estimated by the LTV method. The real image typically contains more surface and normal variations in , while the spoof image appears smoother, lacking 3D depth.
- Column 3: This is the reflection image (ρ Image) estimated by LTV, intended to retain material and texture while separating lighting effects. The estimate can still depend on illumination and decomposition errors. The reflection images of real faces and spoof images often have significant differences in details and textures.
- Column 4: This is the Fourier spectrum of the original image, with the centered spectrum revealing the global spatial frequency distribution, reflecting the image's detail and texture richness.
- Column 5: This is the centered Fourier spectrum after DoG filtering. Compared with the preceding column, high-frequency noise is reduced; the authors also observe differences between the two image types in the directional distribution of mid-frequency components.
Training and Validation Dataset
Data was collected using a regular inexpensive webcam across three different sessions, spaced about two weeks apart, with varying environmental and lighting conditions. A total of 15 participants were involved, with each session capturing live images (500 images, 20fps) and corresponding spoof images.
The spoof images were first captured with a high-quality camera, then printed in different sizes and formats (photo paper and A4 color print) to simulate various attack scenarios. The training set consists of images from the first two sessions (approximately 1743 live and 1748 spoof images), and the testing set comes from the third session (3362 live and 5761 spoof images), with some participants appearing only in the test set.
All images underwent face detection, eye localization, geometric normalization, cropped to 64×64 pixels, and converted to grayscale. Detailed data configuration is shown in the table below:
Discussion
SLR Performance Evaluation
To validate the effectiveness of the aforementioned features and models in real-world applications, the authors first used sparse logistic regression (SLR) to perform recognition experiments on different image features. The left side of the above figure shows the performance of each feature type in the recognition task, with each line representing a feature input. The results are as follows:
- RAW (Raw grayscale images): The performance is the worst, indicating that directly using raw pixel values as input cannot effectively distinguish between live and spoof images.
- LTVu (Lighting image from LTV decomposition): Similarly, it performs poorly. While it contains lighting information, its non-linear characteristics hinder effective classification by a linear classifier.
- LTVp (Reflection image from LTV decomposition): Performance significantly improves, with an AUC of approximately 0.78, indicating that the reflection features capture material and texture differences more effectively.
- DoG (Difference of Gaussian filtered images): It also shows good performance with an AUC of approximately 0.75. Emphasizing mid-frequency features enhances the separability of the image in the feature space.
- HF (High-frequency components): Performance is relatively poor, with an AUC of approximately 0.69, indicating that although high-frequency components can capture details, they are also more susceptible to noise and distortion, leading to instability.
In summary, the LTVp and DoG features outperform RAW and LTVu in recognizing live and spoof images, further confirming the discriminative potential of reflection and mid-frequency features for this task. In contrast, high-frequency information, although capturing some edge features, lags behind LTVp and DoG due to its instability. Therefore, the subsequent experiments no longer use raw images as input features.
Further, the authors analyzed the performance differences between different sparse linear classification models using the same feature (in this case, DoG filtered images), as shown on the right side of the figure:
- Standard Sparse Logistic Regression (SLR): Achieves an AUC of approximately 0.75 with DoG images, serving as the baseline.
- Sparse Low-Rank Bilinear Logistic Regression (SLRBLR): After incorporating the image structure and low-rank assumption, the AUC significantly increases to approximately 0.92.
- SLRBLR Special Case SLRBLRr1 (rank=1): With the projection matrix restricted to rank 1, its AUC reaches 0.95, above SLRBLR and SLR in this DoG experiment.
With the same DoG input, the bilinear models improve both the ROC curve and AUC over standard SLR; the rank-1 case performs best in this comparison. The plot alone cannot tell us which particular direction of variation caused the gain.
From the feature-design perspective, reflection features (LTVp) and mid-frequency features (DoG) perform better under SLR than RAW and LTVu. From the classifier perspective, introducing the low-rank bilinear model further raises AUC with DoG input.
In this dataset's DoG experiment, AUC rises from 0.75 with SLR to 0.95 with the rank-1 bilinear model.
Classifier Performance Evaluation
To further improve recognition performance, the authors extended the original linear classification model to Non-linear Sparse Logistic Regression (SNLR), with the left side of the above figure showing its detection performance with different input features. By introducing a non-linear decision boundary, the overall detection capability is significantly improved.
For example, using the lighting component (LTVu) obtained from LTV decomposition, the paper reports an AUC of about 0.22 for the linear classifier, indicating poor ranking under that training setup. A value below 0.5 does not itself mean that the feature contains no information. However, after applying SNLR, the AUC dramatically increases to 0.92, fully demonstrating that this feature has potential classification value but requires a more expressive non-linear model to uncover it.
Additionally, when the reflection component (LTVp) and lighting component (LTVu) are combined as input, classification performance improves further, with an AUC of 0.94, showing that these two features have complementary information, and their combination enhances the model's discriminative ability.
At low false-positive rates, the DoG-filtered image's ROC curve rises quickly, so the authors consider it a useful practical choice in this comparison. Looking across the entire curve, the fused LTVp and LTVu features also perform well.
The right side of the figure further compares the overall classification accuracy of three classifier models with different feature inputs: Sparse Logistic Regression (SLR), Sparse Non-linear Logistic Regression (SNLR), and Probabilistic Support Vector Machine (SVM). The classification results are based on 9,123 test images, with accuracy calculated by setting appropriate thresholds and determining the proportion of correctly classified samples.
The bars show that DoG with SNLR or SVM is more accurate than DoG with SLR, but LTVu and fused LTV features with SNLR or SVM score higher than DoG in this chart. DoG is therefore not the most accurate feature under every classifier. HF performs much worse with SVM, while with SLR it is close to DoG.
The paper separately reports 76.7% classification accuracy for an earlier Fourier-spectrum method, about 10 percentage points below its DoG and nonlinear-classifier result; that earlier value is not plotted among the bars. The LTV reflectance (LTVp) and illumination (LTVu) images also work well with nonlinear models, and fusing them gives a small further gain.
Taken together, DoG with a nonlinear sparse classifier gives a favorable ROC curve at low false-positive rates; the best overall accuracy in the right-hand chart still depends on the feature and classifier pairing.
Conclusion
This paper uses a physical reflectance model to look for properties of a real face that differ from a photograph. Its image decomposition and feature-learning approach offers an early way to reason about liveness detection and may inform later work on other modalities and lighting conditions.
In practice, the authors propose a non-intrusive, real-time method that uses one ordinary-camera image and no extra hardware. That makes the idea appealing for integration into an existing system.
Liveness detection has since advanced, but the paper remains useful for understanding why image formation and feature choice matter. Its experiments cover photograph attacks involving 15 subjects.