Beyond the Pixel World: A Novel Acoustic-Based Face Anti-Spoofing System for Smartphones
- Publication
- IEEE Transactions on Information Forensics and Security 2022Journal
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@article{faceantispoofingechofas2022,
title = {{Beyond the Pixel World: A Novel Acoustic-Based Face Anti-Spoofing System for Smartphones}},
author = {Chenqi Kong and Kexin Zheng and Shiqi Wang and Anderson Rocha and Haoliang Li},
year = {2022},
journal = {IEEE Transactions on Information Forensics and Security},
url = {https://doi.org/10.1109/tifs.2022.3202115},
doi = {10.1109/tifs.2022.3202115}
}
The application of acoustic technology in the Face Anti-Spoofing (FAS) domain is not extensively covered.
Among the FAS papers we have been reading, we might find only one or two acoustic approaches a year that attract much follow-up discussion.
Defining the Problem
The technological evolution of FAS is like an ongoing excavation of recognition engineering.
We've studied the shadows of LBP, the gradients of HOG, the high-frequency flicker on CNN feature maps, and even tried to find the rhythm of life from the heartbeat signals of rPPG. With each advancement in forgery techniques, we are forced to redefine the evidence of "liveness."
But one thing has always remained unchanged:
- No matter how much we strengthen features, add modalities, or blend loss functions, these methods still heavily rely on "vision."
The more we see, the more precisely we see, the more accurately we observe.
But what about reality?
The lens might have fingerprints, the lighting might suffer from overexposure, and the user's face is not always aligned at the angle desired by the algorithm.
Additionally, almost all RGB-based recognition systems face a few classic problems:
- The domain gap between new phones and old models;
- The spectral difference between indoor lighting and outdoor sunlight;
- The disconnection between data distribution and inference conditions.
To solve these issues, some have turned to hardware enhancements, such as infrared, depth modules, and even millimeter-wave radar.
But these solutions are expensive!
They come with high costs and complex deployments, ultimately shifting the recognition barrier from algorithms to costs and specifications, without truly addressing the need for generalization.
Thus, the author presents a different perspective:
If we can't see clearly, how about we listen?
Speakers and microphones are ubiquitous in smartphones, requiring no additional installation or widespread technical adaptation.
And sound is not just a carrier of information; it can also act as a detector. From a round-trip sound wave path, we can identify geometric information about the face:
- A sheet of paper has a dry, thin echo;
- A screen replay gives back a distorted, unstable reflection.
A real face, too, can leave an acoustic texture shaped by its contours and material. These are cues to learn.
But, is this really useful?
Solving the Problem
Before diving into the system design, the author created a brand new acoustic database: Echo-Spoof.
It is completely different from traditional FAS databases, which mainly focus on images. Echo-Spoof, on the other hand, collects reflection signals from "sound waves."
If we expect to perform anti-spoofing solely with the native speakers and microphones of smartphones, we must ensure that this acoustic detection mechanism remains effective in the real world.
The author set three core requirements:
-
Acoustic-based:
Earlier FAS databases mainly collected face photos or videos. Echo-Spoof instead collects reflected acoustic signals and extracts clues from the face's curvature, contours, and material. It needs no RGB image for this acoustic dataset, which better protects image privacy, though acoustic data may still carry identity or environmental information. A later experiment separately collects synchronized RGB images.
-
Large-scale:
This database comes from 30 volunteers (with an equal gender split), collecting more than 250,000 sound wave signal fragments. A large-scale dataset is necessary to cover the uncontrolled variables found in various real-world scenarios.
-
High-diversity:
In the real world, variability comes from many directions: different phones, different distances, different noise levels, different pitch angles, and so on.
The author specifically designed these conditions to simulate common real-world usage scenarios, such as "the distance between the person and the phone being 25 to 45 cm," "ambient noise fluctuating between 40 dB, 60 dB, and 70 dB," or "facial pitch angles being -10°, 0°, and +10°."
These details were included in the database collection manual, ensuring that Echo-Spoof is not just an idealized small dataset.
The data collection process is shown in the figure below:
Using a smartphone's earpiece speaker, the author emitted designed sound waves and recorded the reflected signals with the top microphone. The figure illustrates how a real face and a mask produce different reflections.
The authors report ethics approval before collection.
Variable Considerations
-
Device: The author selected four Android smartphones (Samsung S9, Samsung S21, Samsung Edge Note, Xiaomi Redmi 7) for data collection. Different smartphones have subtle differences in their speaker and microphone manufacturing processes, which can lead to additional noise distribution and better test the model's cross-device stability.
-
Distance: When using smartphones daily, a person's face is typically about 25 to 45 cm from the screen. The author collected sound waves at 25 cm, 35 cm, and 45 cm to observe the impact of distance on signal-to-noise ratio (SNR).
-
Ambient Noise: In real life, quiet offices and noisy cafés are two entirely different worlds. The author controlled the ambient noise at three levels: 40 dB (quiet), 60 dB (slightly noisy), and 70 dB (very noisy), using a noise detection app to monitor if the levels met the criteria.
-
Pitch: Even the handheld angle can influence the relative position between the face and the phone: pitch angles of -10°, 0°, and +10° were considered. Since face detection in the real world is not always perfectly vertical, small angle variations are more in line with user habits.
With such detailed design, Echo-Spoof has achieved a high level of "scale" and "diversity." This allows for more effective testing and evaluation of the acoustic detection's stability and resistance to interference in different scenarios during later stages.
Signal Design
To ensure that Echo-FAS can operate smoothly in everyday scenarios, the author proposed the following key considerations:
-
Improving Recognition Quality: The author used frequency-modulated continuous waves (FMCW) to capture multiple echoes from different regions of the face, extracting geometric information about the face.
-
Reducing Environmental Interference: By utilizing higher frequency ranges (12–21 kHz) to separate from common noise frequencies (below 8 kHz) and adding a pilot tone for synchronization, the author eliminated timing discrepancies between devices.
-
User Experience: The author aimed to avoid harsh sounds while maintaining enough energy for detection. The Hamming window function was used to suppress unnecessary spectral components, and the volume was controlled to minimize the likelihood of being heard by the user.
The final design is shown in the figure below:
The entire signal lasts approximately 0.8 seconds with a sampling rate of 44.1 kHz (a typical value supported by most Android devices).
A 250-sample, 11.025 kHz pilot is followed by an 8000-sample gap and nine chirps. Three sweeps—12–17, 14–19, and 16–21 kHz—each repeat three times. The 12–21 kHz range describes the chirps, excluding the pilot.
Each chirp consists of 60 samples, and the interval between two chirps is 3000 samples to avoid time-domain overlap.
Signal Analysis
Echo-FAS utilizes FMCW (Frequency-Modulated Continuous Wave) technology, a common frequency-sweeping method used in radar systems originally designed for precise distance measurement. Here, it has been adapted to act as an acoustic scanner for facial geometry recognition.
The principle is not difficult to understand: different parts of the face (such as the nose bridge, lips, forehead) are at different distances from the phone's speaker. As a result, sound waves, when encountering these structures, are reflected back with varying time delays and phase changes. If these subtle differences can be precisely analyzed within a specific frequency band, they can form a set of facial contour features sufficient for recognition.
To explain the theoretical basis for this ability, the author cites the resolution calculation formula in the FMCW framework:
Where is the speed of sound (approximately 343 m/s), is the bandwidth of the sweeping signal, and represents the slope of the frequency variation.
The author estimates this 3.43 cm echo-range resolution from the 5 kHz bandwidth of each individual chirp. It illustrates how bandwidth affects echo separation.
Based on this recognition requirement, the author chose a frequency range of 12 to 21 kHz.
Higher frequencies avoid some low-frequency ambient noise, but 12–21 kHz and the pilot are not universally inaudible. Hearing depends on the individual, age, and volume.
The authors’ preliminary tests found degraded phone performance above roughly 20 kHz. The chosen range balances audibility, noise, and hardware response rather than ensuring equivalent operation on every phone.
To ensure the signal can be accurately recognized in recordings, the author introduced an 11.025 kHz pilot tone at the beginning of the sound wave. This is a continuously stable pure tone that is distinct from other chirp signals and environmental noise in the spectrum.
By performing cross-correlation with the recording, the system locates the signal's starting point and estimates the timing delay between the speaker and microphone for subsequent echo extraction.
For the design of the frequency sweep signals, the author set the length of each chirp to 60 samples (approximately 1.4 milliseconds) and added a 3000-sample interval between chirps.
This configuration considers both physical and engineering limitations:
If the chirp is too long, it might improve the signal-to-noise ratio but could also overlap with the facial echo in time; for example, at a distance of 30 cm, the round-trip time for sound is about 1.7 milliseconds, which corresponds to 77 samples. If the chirp length exceeds this value, it becomes difficult to distinguish the echo from the direct signal.
Conversely, if the interval between two chirps is too short, the echo from a distant object (like a wall) could fall within the range of the next chirp's facial echo, causing aliasing and interference.
The 3000-sample gap is about 68 ms. With the pilot and waiting interval, the emitted sequence lasts about 0.8 seconds. This is sensing-signal duration, not neural-network latency or a universal acceptability threshold.
Although the frequency design avoids the audible range as much as possible, some signals still fall below 17 kHz, potentially causing auditory interference for certain users.
To address this, the author incorporated a Hamming window in the signal design, ensuring that the start and end of the chirp have a smooth envelope shape, further reducing sound sharpness. They also conducted volume optimization tests for various phone models to ensure the output volume balanced "distortion-free but sufficiently loud."
Over 90% of volunteers reportedly experienced little annoyance. Audibility and responses across ages and devices still need separate measurement.
Signal Processing
If designing a sound is like a language, then signal processing is learning to recognize its meaning.
In Echo-FAS, the recorded sound is not just an echo; it is a mixture that includes the direct transmission signal from the speaker, the primary echo from the face, and background reflections from other objects in the environment.
The purpose of signal processing is to extract the true face geometry from this collection of sound waves, which will then be used as input for the subsequent recognition model.
The entire processing flow is divided into three stages:
- Segmentation
- Direct Transmission Removal
- Target Reflection Extraction
As shown in the figure below:
First, signal segmentation.
The recording starting point is not stable, as the speaker and microphone on the phone are not perfectly synchronized. Therefore, the author designed a 11.025 kHz pilot tone as a synchronization marker. Through cross-correlation with the recording content, the system can accurately determine the true starting point of the signal.
After locating the start and filtering out ambient noise below 12 kHz, the recording is segmented into nine parts, corresponding to the nine chirps designed earlier. Each segment contains a mixture of direct transmission and various echoes, serving as the first step for rough localization.
Next, direct transmission removal.
This signal, emitted by the speaker and directly received by the microphone without reflection, typically dominates the highest peak due to its short travel distance and relatively high energy. To remove it, the author applies a matched filter using the original chirp, identifies the direct transmission's position, and discards the samples up to and including that signal. The later samples may contain facial or background echoes.
Finally, the critical step of target reflection extraction.
After removing the direct transmission signal, the remaining signal might still contain echoes from distant objects like walls or tables. These echoes occur later and are more dispersed. However, facial echoes are usually the shortest in distance and appear "earliest and most concentrated" in each segment, as shown in the figure below:
Thus, the author proposed an adaptive localization algorithm: for each segment, a matched filter is applied to find the peak with the highest energy, and the peak positions from the nine segments are averaged to find the "most consistent and likely" position that represents the facial echo. The algorithm iteratively searches for the average point with the smallest standard deviation, ensuring stable localization. Then, 60 samples from each segment after this point are extracted as the final output of the facial acoustic features.
As a result, each detection extracts nine 60-point sound wave blocks from the original recording, which serve as the basis for the subsequent model to judge authenticity. No additional sensor or multi-channel microphone array is required: the phone's speaker emits the designed signal, and its top microphone records the echoes.
Model Architecture
After solving the acoustic challenges, we now move on to the model architecture, which should be relatively familiar to our readers.
The diagram below illustrates the entire recognition process, divided into three stages:
- Preprocessing
- Two-Branch Feature Extraction
- Cross-Attention Fusion Decision
For our readers, this section should be relatively straightforward, so let's take a closer look at each stage.
Preprocessing
The input is a raw sound signal recorded from the microphone.
Using the signal processing steps introduced earlier, the system first synchronizes the signal, removes the direct transmission signal, and extracts nine segments of the echo from the entire recording.
Two-Branch Architecture
These nine echo segments are fed into two parallel and complementary neural network branches for the analysis of two types of frequency features:
-
Global Frequency Branch (right side, Transformer branch):
Each signal segment undergoes a Fast Fourier Transform (FFT) to convert it into a frequency-domain representation; the nine spectra become nine input tokens for the Transformer.
This branch focuses on capturing the distribution of the signal’s frequency and long-range dependencies. The Transformer learns the "frequency patterns" associated with facial structures from these spectra, creating a set of global perspective feature maps.
-
Local Frequency Branch (left side, CNN branch):
The other side uses Short-Time Fourier Transform (STFT) to convert each signal segment into a time-frequency spectrogram.
These spectrograms retain how frequency changes over time as the emitted sound sweeps and reflects from the face. The CNN then learns local patterns and details from that response, extracting a texture of geometric clues rather than measuring a heartbeat or facial motion.
These two paths offer distinct modeling advantages: the Transformer focuses on the overall frequency structure, while the CNN focuses on local response textures. In Echo-FAS, they are designed to complement each other, not oppose one another.
Each branch outputs a set of feature representations: the global features and the local features .
To align these two perspectives effectively, Echo-FAS incorporates a Dual Cross-Attention Module. As shown in the diagram (d), the Cross-Attention module allows to attend to , and vice versa, allowing the two feature sets to mutually attend and merge. Eventually, these two sets of features are concatenated and passed to the classification layer for the final decision.
Depth Selection
To validate the rationality of this architecture, the author conducted a series of ablation experiments.
The Transformer-block experiment is on the left: EER is lowest at 10 blocks. The CNN-layer experiment is on the right: EER is lowest at 5 layers. The author chose these depths for the two branches.
Discussion
Cross-Identity Recognition Capability
In real-world scenarios, FAS systems cannot pre-recognize all users. They must be able to make correct judgments when encountering "unseen faces." Therefore, the first experiment focuses on the core generalization challenge: cross-identity liveness detection.
In the table above, the author trained Echo-FAS on 25 users and tested it on 5 additional subjects who had never appeared in the training data.
The results show that Echo-FAS achieved 98.79% AUC and 95.18% ACC under this setting, demonstrating strong performance. The table also includes other classifiers, including MLP, CNN, and Transformer architectures; Echo-FAS has the best AUC, ACC, HTER, ACER, and EER among the methods listed.
This shows that acoustic signals offer distinguishable cues in this cross-identity test and supports the design of the two-branch fusion architecture.
The 25-person training and five-person test support transfer across identities in this setting.
In short, Echo-FAS still distinguished live faces from spoofs among people it had not seen during training. Cross-device and cross-attack behavior each require their own tests.
Stability Under User Habits
Not everyone holds their phone in an ideal position.
Some users tilt their head down to scroll, others unlock with a side profile, and some habitually use the phone at a slight upward angle. These daily habits, which were often seen as "non-ideal poses" in previous FAS systems, are actually the default in real-world usage.
To test the stability of Echo-FAS under these operational variations, the author designed a "user pitch tolerance test":
- The angle between the face and the phone (pitch) was divided into three settings: -10°, 0°, and +10°, corresponding to common usage scenarios such as tilted, frontal, and downward-facing angles, respectively. The model was tested for each angle.
Across these three pitches, Echo-FAS reached 97.97%–98.40% AUC and 5.71%–8.25% HTER. Both metrics beat the baselines listed in the table at each tested angle. The results support some stability under these small pitch changes.
For a user, this means the phone need not be held at exactly one face-on angle within the tested ±10° range.
The acoustic geometry is more than the language of the system design: this experiment tests a limited variation in handheld angle. Larger pose changes are outside the table's scope.
Multi-Modal Fusion
In many real-world application scenarios, we cannot predict which phone a user will use or under which lighting conditions the identification will take place. This makes RGB-based FAS models prone to domain gaps caused by device heterogeneity, leading to significant performance drops in deployment.
Echo-FAS's acoustic modality offers information about facial reflections that differs from image appearance and may complement visual features. It is still affected by differences between phones' speakers and microphones.
To verify the complementarity between sound and image, the author designed a multi-modal fusion experiment, as shown below:
The authors separately collected synchronized acoustic signals and face images from 10 volunteers. Each multimodal sample pairs a sound recording with its corresponding face image.
The acoustic signal is processed by the Echo-FAS backbone to extract feature , while the image is processed by a ResNet18 pre-trained on ImageNet to extract visual features . After fusion, the two features jointly make a decision.
The experiment used four smartphones (Samsung S9, S21, Edge Note, and Redmi7) for data collection. For each training session, three devices were used, with the remaining device used as the test domain, simulating the cross-device generalization challenge encountered in real deployment.
The results of the experiment are shown in the table below:
Across all four held-out phones, fusion has higher AUC and lower HTER than RGB alone. Mean AUC rises from 93.48% to 99.08%, while mean HTER falls from 18.26% to 4.34%. Acoustic-only mean AUC is 83.89%; on the Note device it is just 68.11%, so the acoustic model itself also shows a cross-device gap.
In this cross-device experiment, combining acoustic and image features improved the test results. It does not show that the acoustic model alone has solved device differences.
Conclusion
Echo-FAS offers a practical direction for FAS.
It uses a short sensing signal and sensors already built into the tested phones. Users need not learn a new gesture or rely on a particular lighting setup. In the tested range of distances and pitch angles, the system can listen for another kind of evidence without asking them to hold still through a long capture.
That ease has a condition: the phone needs both a suitable speaker and microphone, along with a working emission, recording, and synchronization path. The roughly 0.8-second signal is a sensing duration; deployment speed and compatibility on other phones still need validation.
The system also has limits. The paper mainly tests 2D presentation attacks; how it fares against complex 3D masks remains open. Different hardware leaves different acoustic fingerprints, and filtering and adaptive localization cannot erase every frequency-response gap. Sound is still shaped by the machine carrying it.
So Echo-FAS need not replace visual FAS to be useful. It offers a lightweight, complementary sensing modality, as the fusion results show, even though acoustic-only transfer across devices remains uneven. When images are unreliable, it offers another possibility:
The clue to a spoof may come not only from an image, but from an echo.