Back to paper library
ACM IMWUT 2024Face anti-spoofingPaper notes

AFace

The Boundary of Waves

Original paper

AFace: Range-flexible Anti-spoofing Face Authentication via Smartphone Acoustic Sensing

Paper authors

Zhaopeng Xu · Tong Liu · Ruobing Jiang · Pengfei Hu · Zhongwen Guo · Chao Liu

Cite this paper · BibTeX

The citation identifies the linked source version. It cites the original paper and its authors.

@article{faceantispoofingaface2024,
  title = {{AFace: Range-flexible Anti-spoofing Face Authentication via Smartphone Acoustic Sensing}},
  author = {Zhaopeng Xu and Tong Liu and Ruobing Jiang and Pengfei Hu and Zhongwen Guo and Chao Liu},
  year = {2024},
  journal = {Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
  url = {https://doi.org/10.1145/3643510},
  doi = {10.1145/3643510}
}
On this page

Opening this paper, we found it has 33 pages!

We couldn’t help but feel a headache coming.

Defining the Problem

Over the past decade, smartphone authentication systems have evolved from PINs, fingerprints, iris scans to face recognition.

These methods build progressively upon one another, shifting from memory-based to biometric-based, from contact sensing to contactless sensing, each step aiming for higher convenience and security.

Yet this technological evolution seems to have hit certain boundaries.

  • PIN codes are easy to forget and vulnerable to shoulder surfing;
  • Fingerprint recognition has uniqueness but can be replicated with silicone molds;
  • Iris recognition is accurate but costly, limiting its widespread adoption;
  • Face recognition is widely deployed but fragile against basic photo and video attacks.

Even with further integration of 3D sensing modules, such as dot projectors and infrared structured light, there remain three major risks:

  1. Cost and size constraints: These hardware components often increase the phone’s BOM by over 5%, hindering mass adoption;
  2. Resource and latency bottlenecks: High-performance models often require over 4 seconds for inference, which is unacceptable for users;
  3. Forgery and privacy risks: Visual information can not only be replicated but is also prone to leakage, making defense difficult.

Moreover, mainstream anti-spoofing research continues to advance—from texture features to deep learning—but it remains trapped by a core challenge:

When attackers hold a 3D printed model, existing visual systems still cannot identify the material nature.

Therefore, some have begun to explore signal-based approaches.

Acoustic waves, WiFi, RFID, and RF signals have started to appear in authentication literature. These non-visual methods attempt to penetrate appearance, reading user gait, breathing, heartbeat, and reflection characteristics. However, these approaches face their own limitations:

  • Gait features are overly influenced by clothing and posture;
  • WiFi and RFID are accurate but rely on additional devices and tags;
  • Ear canal recognition requires uncomfortable in-ear equipment.

Even recent acoustic-based face recognition achieved with smartphone microphones and speakers mainly focus on shape reconstruction, still vulnerable to imitation attacks using 3D fake face models. Once the shape matches, echoes are identical and recognition fails.

So the question arises:

If echoes can be mimicked, is there any feature that attackers simply cannot forge?

Solution

AFace is an acoustic-based authentication system.

It relies on a pair of built-in smartphone components—speaker and microphone—to perform face recognition.

The overall process includes four steps:

  1. Acoustic Wave Emission: During enrollment, the user positions the phone at two different distances, allowing it to emit acoustic waves and receive reflected echoes, thereby establishing a mapping between “energy and distance.”
  2. Signal Preprocessing: The echo signals undergo alignment, noise cancellation, and matched filtering to suppress direct-path signals and background reflections, extracting information related to distance.
  3. Compensation Model: To handle variations in the user-to-phone distance, the system builds a Range Adaptive (RA) model and computes an Energy Compensation Function (ECF) using those two enrollment distances. It compensates later echoes to a reference distance while preserving energy differences caused by materials.
  4. Feature Extraction and Model Training: After signal compensation, key features are extracted, and a deep learning model is trained for classification to serve as the basis for subsequent matching.

Acoustic Wave Emission

Sound must be both emitted and recognized; energy must be focused yet not disturb the user.

To ensure the signal is both “solvable” and “practical,” the authors designed several aspects:

  1. Signal Design: AFace adopts FMCW (Frequency Modulated Continuous Wave) as the signal foundation, with the following advantages:

    1. High Frequency Separation: The 10–22 kHz band avoids most ambient noise (< 8 kHz), allowing high-pass filtering to remove environmental interference;
    2. Sufficient Resolution: A 12 kHz bandwidth corresponds to 1.4 cm distance resolution, capable of discerning subtle differences across facial regions;
    3. Smartphone Compatibility: Most smartphones support frequencies below 24 kHz; considering attenuation, 22 kHz is a reasonable upper limit;
    4. Non-intrusive Sound: Signal duration of 10 ms plus 40 ms intervals make the user “barely aware” of the sound;
    5. Energy Control: Using a Hanning window focuses energy near 16 kHz, avoiding spectral leakage.

    The mathematical form of this signal is:

    s(t,t^m)=rect(t^Tp)⋅ej2π(fct+12kt^2)s(t, \hat{t}_m) = \text{rect}\left( \frac{\hat{t}}{T_p} \right) \cdot e^{j 2\pi \left( f_c t + \frac{1}{2} k \hat{t}^2 \right)}

    No need to fully grasp the formula; the components mean:

    • rect(⋅)\text{rect}(\cdot): restricts the signal duration to 10 ms (Tp=10T_p = 10 ms);
    • fcf_c: carrier frequency (centered at 16 kHz);
    • kk: chirp rate;
    • t^=t−tm\hat{t} = t - t_m: fast time used to locate the time window for each segment;
    • tm=mTt_m = mT: slow time, representing the emission time point of each chirp (m=0,1,2,...m = 0, 1, 2, ...).

    With this design, the signal physically propagates, reflects, and converges into a set of digital features suitable for comparison.

  2. Iso-depth Model: After sound emission, the microphone actually receives a mixture of signals. To analyze these, the authors classify them into four categories:

    1. Environmental noise;
    2. Direct path signals (from speaker directly to microphone);
    3. Reflections from the face (the target);
    4. Reflections from other objects (to be removed).

    Accordingly, AFace establishes a multipath reception model:

    sr(t,t^m)=∑irect(t^−τiTp)⋅ej2π(fc(t−τi)+12k(t^−τi)2)s_r(t, \hat{t}_m) = \sum_i \text{rect}\left( \frac{\hat{t} - \tau_i}{T_p} \right) \cdot e^{j 2\pi \left( f_c (t - \tau_i) + \frac{1}{2} k (\hat{t} - \tau_i)^2 \right)}

    where

    • τi\tau_i: delay of the ii-th path;
    • modeled as a delayed version of the emitted signal.

    To simplify processing, the overall signal is further decomposed into:

    sr(t,t^m)=∑isi+∑jsjs_r(t, \hat{t}_m) = \sum_i s_i + \sum_j s_j
    • sis_i: target reflections (face);
    • sjs_j: non-face reflections (to be removed).

To associate acoustic waves with a “face,” the core technique AFace uses is matched filtering, which performs conjugate convolution of the received echo signal with a reference signal srefs_{\text{ref}}, finding the temporal overlap region to form an energy peak:

E=∑isi⊙srefE = \sum_i s_i \odot s_{\text{ref}}
  • ⊙\odot: the conjugate convolution used for matched filtering;
  • The energy peak indicates the alignment between echo and reference signal, i.e., the distribution of reflection distances.

Because regions with close distances cause overlapping peaks and energy superposition, this “superposition effect” serves as the basis for differentiating facial region reflections.

To equip the model with “facial awareness,” AFace divides the face into six iso-depth zones, as shown below:

  • Nearest: Nose tip (small reflection area, low energy);
  • Front ring: eye sockets, forehead, lips (large reflection area, facing phone, highest energy);
  • Side cheek: oblique reflections, limited energy;
  • Farthest: ears (far distance, complex structure, lowest energy).

These physical conditions translate into model peak energy distributions as in figure (b). The authors also observed changes in echo energy when they covered facial regions in later experiments, supporting this way of dividing the face into zones.

Signal Preprocessing

Every signal tells a story, but first, we need to silence the noise.

While signal design is crucial, what actually reaches the receiver is a distorted, delayed, and noisy acoustic wave. We cannot assume that the emitted signal from the phone is exactly what we receive, because hardware imperfections and uncontrollable environments turn the "echo" into raw data that must be cleaned and calibrated.

AFace’s preprocessing module consists of four steps:

  1. Acquiring the True Reference Signal

    Smartphone speakers and microphones have different gains across frequency bands, causing emitted and received signals to be non-ideal.

    To address this, the authors propose a practical “hardware removal” approach:

    • Collect background signals once in an empty environment;
    • Then place a 1 cm×1 cm1 \text{ cm} \times 1 \text{ cm} paperboard at 15 cm distance to collect a single reflection;
    • Taking the difference approximates the pure single echo from the paperboard, serving as the reference signal srefs_{\text{ref}}.

    The significance is that we no longer assume knowledge of the emitted signal but instead infer the true acoustic signal through measurement.


  1. Aligning Signal Start Point

    In the paper’s phone implementation, the starting times of speaker output and microphone recording cannot be directly synchronized, so the received signal cannot be directly timestamped to the “true emission time.”

    To solve this, the authors leverage the phone’s internal “solid transmission” from speaker to microphone:

    • Sound travels only a few centimeters inside the phone (speaker → microphone) with about 0.04 ms delay;
    • Solid transmission suffers low loss and strong signal strength;
    • Using cross-correlation, the direct path signal arrival can be precisely detected as the time reference.

    This introduces about 1–2 samples of error (about 0.7 cm), but it does not affect relative facial distances in the same way as absolute range, and the subsequent RA algorithm helps compensate for distance variation.


  1. Noise Removal

    Noise comes in two types:

    • Environmental noise: mostly below 10 kHz and removable by high-pass filtering;
    • Signal noise: e.g., direct-path interference or multipath reflection noise, requiring subtraction using the reference signal.

    Direct subtraction cannot completely remove noise since echo strength attenuates with reflector distance, making waveforms similar but amplitudes different.

    Therefore, the authors introduce a scaling factor ε\varepsilon to minimize the difference error:

    min⁡ε∥Sr−ε⋅Sc∥2\min_\varepsilon \left\| S_r - \varepsilon \cdot S_c \right\|^2

    where

    • SrS_r: received signal;
    • ScS_c: previously recorded clean reference signal;
    • ε\varepsilon: scaling factor solved by least squares.

    The paper shows how ε\varepsilon changes over 200 frames; the previously recorded signal is scaled by this coefficient before subtraction.


  1. Match Filtering

    Once aligned and noise-reduced, we obtain an approximate echo signal sis_i corresponding to the face.

    Next, matched filtering is performed by conjugate convolution of sis_i with the reference signal:

    yt=si⊙sref=sin⁡(πkTpt)πkt⋅rect(t2Tp)⋅ej2πfcty_t = s_i \odot s_{\text{ref}} = \frac{\sin(\pi k T_p t)}{\pi k t} \cdot \text{rect}\left(\frac{t}{2T_p}\right) \cdot e^{j2\pi f_c t}

    This result resembles a sinc function, representing:

    • Reflections at different distances produce energy peaks at different time points;
    • Each peak corresponds to a "depth echo" of different facial regions;
    • Via Fourier analysis, each echo segment maps to frequency:
    fi=−k⋅2ΔRcf_i = -k \cdot \frac{2\Delta R}{c}

    where ΔR\Delta R is the distance difference to the reference signal and cc is the speed of sound.

In other words, through match filtering and Fourier transform, a blurred acoustic signal is restored into a high-resolution “distance spectrum,” representing the depth contour of the face structure.

Range-Adaptive (RA) Compensation Algorithm

Next, we need to convert the varying distances into stable features for reliable face recognition.

Acoustic echoes are functions of distance.

However, in real scenarios, users won’t always hold the phone at exactly 15 cm or 20 cm; the face-to-phone distance fluctuates slightly during each authentication. These distance variations cause signal phase shifts and energy changes, affecting model stability.

To address this, AFace designs a Range Adaptive (RA) compensation mechanism that maps every input back to a stable distance and energy baseline.

This mechanism contains two submodules:

  1. Range Compensation: compensates for phase changes in echo signals;
  2. Energy Compensation: corrects the relationship between reflection energy and distance.

During enrollment, users register at two distances (e.g., D0D_0, D1D_1), recording their nearest echo points (d0d_0, d1d_1) and corresponding maximum energies (e0e_0, e1e_1) as compensation baselines.

During authentication, if face distance is did_i, the echo signal phase is:

φ1=2πfc(t−2dic)+πk(t−2dic)2\varphi_1 = 2\pi f_c \left(t - \frac{2d_i}{c}\right) + \pi k \left(t - \frac{2d_i}{c}\right)^2

The ideal reference phase at d0d_0 is:

φ0=2πfc(t−2d0c)+πk(t−2d0c)2\varphi_0 = 2\pi f_c \left(t - \frac{2d_0}{c}\right) + \pi k \left(t - \frac{2d_0}{c}\right)^2

Their difference:

Δφ=−4πfcc(di−d0)−4πktc(di−d0)+4πkc2(di2−d02)\Delta \varphi = -\frac{4\pi f_c}{c}(d_i - d_0) - \frac{4\pi k t}{c}(d_i - d_0) + \frac{4\pi k}{c^2}(d_i^2 - d_0^2)

This phase difference not only shifts time but also causes peak shifts after Fourier transform, impacting distance estimation and feature extraction.

To align signals, AFace compensates all signals as:

s′=e−jφ1e−jΔφ=e−j(φ1−Δφ)s' = \frac{e^{-j\varphi_1}}{e^{-j\Delta \varphi}} = e^{-j(\varphi_1 - \Delta \varphi)}

That is, restoring the signal phase to the “standard distance,” as if all faces were compared at the same position.

Besides phase offset, echo energy changes with distance and material.

AFace finds this is not merely an error source but a valuable identification feature.

They designed experiments:

  • Move different materials (paperboard, wood, iron) steadily toward the phone;
  • Play signals for 10 seconds each, recording peak energies at various distances;
  • Filter noise and background reflections, perform match filtering to get reflection intensities;
  • Extract maximum energies per distance and plot energy-distance curves;
  • Fit linear regression between energy and distance, yielding high correlation (R2=0.9536R^2 = 0.9536, RMSE = 0.03198).

The diagram below shows how RA maps echo range and energy to the reference position; the energy–distance curves from the material experiment appear in Figure 8 of the original paper.

Within the tested range, energy and distance show a strong linear relationship, while different materials may have different reflection energies.

AFace builds an Energy Compensation Function (ECF):

ECF(di)=e1−e0d1−d0(di−d0)+e0\text{ECF}(d_i) = \frac{e_1 - e_0}{d_1 - d_0}(d_i - d_0) + e_0

This predicts the theoretical energy at distance did_i. The measured energy eie_i is compensated as:

ei′=ei⋅e0ECF(di)e'_i = e_i \cdot \frac{e_0}{\text{ECF}(d_i)}

This compensation doesn’t forcibly normalize eie_i to e0e_0 but preserves its characteristics. In other words:

If a covering material reflects sound differently from real skin, its compensated energy may still differ from the real user’s, even at the same distance.

This “difference-preserving compensation” corrects for distance influence while leaving material-related reflection differences available as a recognition clue.

Feature Extraction

Once signals are aligned, compensated, and filtered, AFace extracts information to distinguish one person from another.

The authors design features in two categories:

  • Distance Features
  • Energy Features

Starting with distance features:

Building on the Iso-depth model, AFace divides the face into six major zones, choosing five for depth analysis:

  • Nose tip
  • Nose root
  • Lips & eye socket
  • Cheek to forehead
  • Side of face
  • Ear is set as the farthest zone with depth baseline zero

Features designed are:

  1. Five absolute distances: estimated from peak positions after match filtering;

  2. Ten relative distance differences: depth differences between any two zones, e.g., nose tip to nose root representing “nose bridge height”;

  3. Five approximate area measures: when adjacent zones have close depths, match filtering produces overlapping peaks whose widths indicate reflector sizes;

    • Specifically, peak widths at 50% energy level (similar to Full Width Half Maximum);
    • Especially measured for nose root and lips–eye socket zones.

Next, energy features:

Different materials reflect sound differently. The authors conducted masking experiments shown below:

By covering different face zones, they observed energy changes. This figure compares an uncovered face with a sheet mask and localized coverings; it does not directly compare a real face with a 3D-printed model.

In the tested coverings, energy could thus differ even when depth features were similar.

The final selected energy features are:

  1. Maximum energy: overall reflection strength reflecting skin and area info;
  2. Average peak energy: average reflection across face;
  3. Variance: indicates how concentrated or dispersed facial depth distribution is;
  4. Peak factor: strength of high reflection points;
  5. Average frequency: main frequency distribution of echoes;
  6. Energy density: concentration of energy within regions.

These collectively reflect material traits, complementing shape-based features.

AFace uses 26 engineered features in total, fusing shape, material, and reflection behavior. The classifier also receives 40 samples of the echo signal, making 66 input values altogether.

Classifier Architecture

After completing feature design and extraction, AFace requires a classifier to convert these features into a “recognize you” decision.

The authors split this into two parts:

  1. Using BiLSTM to capture temporal logic and zonal features within the signals;
  2. Employing iCaRL to address the “forgetting problem” when updating users and expanding classes.

In AFace, the input features are not mere tabular data but complex features including “signal slices,” “distance relations,” and “energy profiles.” To handle this, AFace uses a four-layer BiLSTM structure:

  • 40-dimensional signal slices (time series):

    • Extracted from denoised and distance-compensated echo residuals;
    • Aligned with nose tip as start point, taking 40 samples ≈ 14 cm range;
  • 20-dimensional distance features:

    • 5 absolute depths (nose tip, nose root, lips & eye socket, cheek to forehead, side face);
    • 10 relative depth differences;
    • 5 peak widths after match filtering (reflecting area overlap and shape);
  • 6-dimensional energy features:

    • Reflecting material properties, distribution, and reflection strength (see previous chapter).

BiLSTM parameters:

  • Layers: four layers total;
  • Activation: tanh (hidden states range in [−1,1][-1,1]);
  • Gating function: sigmoid, controlling information retention;
  • Weight initialization: Xavier initialization (optimized for tanh);
  • Forget gate: initialized near 1 (unit-forget-gate), enhancing long sequence learning.

Fully connected (dense) and classifier layers:

  • Dense layer: 128 units with LeakyReLU activation;
  • Dropout layer: rate = 0.5, to prevent overfitting;
  • Softmax layer: outputs probabilities for 40 classes;
  • Classifier: maps softmax probabilities to identity classes.

Another part is the iCaRL incremental learning strategy.

To handle long-term challenges of “adding users” and “face changes,” AFace adopts iCaRL (Incremental Classifier and Representation Learning), allowing the neural network to update as new data arrives.

Incremental learning has two scenarios:

  • Class Increment:

    • When new users register, features are extracted from their echo data to generate new training samples;
    • A new fully connected classification node is added to support the new class;
    • The existing fully connected layers are frozen while the new classification node is trained;
    • Only the new layer is fine-tuned to speed training and stabilize output.
  • Data Increment:

    • When acoustic authentication fails but the user passes backup verification (e.g., PIN input), the system records a possible change;
    • Each “acoustic verification fail but fallback success” stores a new sample;
    • Upon collecting ≥ 5 samples, a “data increment” fine-tuning process triggers;
    • Model architecture remains fixed; fine-tuning uses low learning rate for adaptation and memory update.

This incremental learning aims to reduce “catastrophic forgetting” when new data arrives and to avoid retraining the whole network each time.

The design goal is for AFace to continuously observe and update user memories, conceptualizing the classifier as an adaptive relationship rather than a fixed model.

Discussion

The paper contains numerous experimental tables that we won’t list exhaustively here; interested readers can consult the original publication.

Below, we highlight several key aspects to evaluate the method’s performance.

Different Speaker and Microphone Combinations

Results show the best combination is the top speaker + top microphone.

Using bottom speakers or microphones results in echoes mostly coming from the user’s chest area, which is heavily influenced by clothing material. Even small changes (like changing a jacket) significantly degrade recognition performance.

Hardware design should prioritize top speaker-microphone placement to reduce irrelevant reflections and focus on facial signals.

2D vs. 3D Attacks

In the paper’s single-user experiment, the defense success rate against photo and video attacks was 100%. The tested 3D attacks used three kinds of face covering, for which energy features were also needed. Flat images lack the facial depth echoes, while those coverings also call for comparing reflected energy.

Without energy features, the defense success rate against those three coverings was 2.61%; with them, it reached 98.41%. This supports the role of reflected energy for these materials.

Importance of the RA Algorithm

To verify RA’s effectiveness, the authors had users register at 25 cm and 50 cm, then tested at distances of 25, 30, 35, 40, 45, and 50 cm in single-user mode multiple times.

Without RA, accuracy is higher (>96%) at the enrolled distances (25 and 50 cm), dropping to about 90% elsewhere. With RA enabled, it stays above 95% at all six tested distances, averaging 96.7%. This supports improved distance adaptation within the 25–50 cm range.

Long-Term Use and Facial Changes

The authors evaluated iCaRL incremental learning over two months, observing facial changes and system accuracy. Forty participants were split into two groups of 20, one with iCaRL and the other without. The paper’s prose says data were collected on days 1, 10, 30, and 60, while the imported Figures 23–24 label the second point as day 7.

Changes such as hairstyle, glasses, or makeup may affect echoes. Accuracy declined over time in the group without incremental learning, while the group with iCaRL maintained higher accuracy at the four measurement points. Because the groups comprised different people, this comparison supports the value of updating over time but cannot rule out group differences or isolate any one facial change.

Conclusion

Since 2019, acoustic signals have gradually been introduced into the FAS domain, offering a distinct path from image-based recognition.

However, these methods commonly suffer many limitations, the most critical being:

Acoustic signals alone cannot solve the 3D mask problem!

Masks and real faces produce nearly indistinguishable echoes acoustically.

In AFace, by incorporating the concept of material, acoustic echoes become a composite function of distance + material, finally making acoustic-based FAS feasible.

AFace not only improves 3D attack resistance but also expands real-world applicability through range compensation (RA) and energy modeling (ECF). It is among the few acoustic verification systems fully implemented on commercial smartphones and augmented with incremental learning to handle long-term variation.

The problem remains open, but we now know:

Acoustic signals can stand alone as a reliable biometric clue.