{"notes":[{"id":"face-antispoofing/faceshield","category":"face-antispoofing","legacyPath":"face-antispoofing/2505-faceshield","legacySlug":"2505-faceshield","legacyPaths":[],"title":"FaceShield","subtitle":"臉的庇護所","author":"Z. Yuan","paperDate":"2025-05-14","publishedOn":"2025-06-20","paperTitle":"FaceShield: Explainable Face Anti-Spoofing with Multimodal Large Language Models","paperUrl":"https://arxiv.org/abs/2505.09415","bibliography":{"title":"FaceShield: Explainable Face Anti-Spoofing with Multimodal Large Language Models","authors":["Hongyang Wang","Yichen Shi","Zhuofu Tao","Yuhao Gao","Liepiao Zhang","Xun Lin","Jun Feng","Xiaochen Yuan","Zitong Yu","Xiaochun Cao"],"url":"https://arxiv.org/abs/2505.09415","pdfUrl":"https://arxiv.org/pdf/2505.09415v1","codeUrl":"https://github.com/Why0912/FaceShield","preprint":{"id":"2505.09415","submittedOn":"2025-05-14","url":"https://arxiv.org/abs/2505.09415"},"publication":{"venue":"AAAI","year":2026,"kind":"conference","url":"https://ojs.aaai.org/index.php/AAAI/article/view/37945"},"citation":"@misc{faceantispoofingfaceshield2025,\n  title = {{FaceShield: Explainable Face Anti-Spoofing with Multimodal Large Language Models}},\n  author = {Hongyang Wang and Yichen Shi and Zhuofu Tao and Yuhao Gao and Liepiao Zhang and Xun Lin and Jun Feng and Xiaochen Yuan and Zitong Yu and Xiaochun Cao},\n  year = {2025},\n  eprint = {2505.09415},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2505.09415}\n}\n","citationPath":"/content/papers/citations/face-antispoofing/faceshield.46ea2b7a56fa.bib"},"thumbnail":{"src":"/content/papers/assets/72227f25d8bae3bb37fab6ee6e697210da60a8f59ddb17f11dd2634a8bb762df.jpg","width":1436,"height":560},"locale":"zh-hant","path":"/content/papers/zh-hant/face-antispoofing/faceshield.f96e66b80226.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"模型架構","text":"模型架構","depth":3},{"id":"spoof-aware-vision-perceptionsavp","text":"Spoof-Aware Vision Perception（SAVP）","depth":3},{"id":"prompt-guided-vision-token-maskingpvtm","text":"Prompt-Guided Vision Token Masking（PVTM）","depth":3},{"id":"模型流程","text":"模型流程","depth":3},{"id":"二階段訓練","text":"二階段訓練","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"intra-dataset-評估","text":"Intra-dataset 評估","depth":3},{"id":"cross-dataset-評估","text":"Cross-dataset 評估","depth":3},{"id":"消融實驗","text":"消融實驗","depth":3},{"id":"可視化分析","text":"可視化分析","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"face-antispoofing/i-fas","relation":"comparison","topic":"用多模態語言模型判斷人臉偽造"},{"id":"face-antispoofing/instructflip","relation":"comparison","topic":"用多模態語言模型判斷人臉偽造"}]},{"id":"face-antispoofing/i-fas","category":"face-antispoofing","legacyPath":"face-antispoofing/2501-i-fas","legacySlug":"2501-i-fas","legacyPaths":[],"title":"I-FAS","subtitle":"分類的終章","author":"Z. Yuan","paperDate":"2025-01-03","publishedOn":"2025-06-06","paperTitle":"Interpretable Face Anti-Spoofing: Enhancing Generalization with Multimodal Large Language Models","paperUrl":"https://ojs.aaai.org/index.php/AAAI/article/view/33073","bibliography":{"title":"Interpretable Face Anti-Spoofing: Enhancing Generalization with Multimodal Large Language Models","authors":["Guosheng Zhang","Keyao Wang","Haixiao Yue","Ajian Liu","Gang Zhang","Kun Yao","Errui Ding","Jingdong Wang"],"url":"https://ojs.aaai.org/index.php/AAAI/article/view/33073","pdfUrl":"https://ojs.aaai.org/index.php/AAAI/article/download/33073/35228","codeUrl":null,"preprint":{"id":"2501.01720","submittedOn":"2025-01-03","url":"https://arxiv.org/abs/2501.01720"},"publication":{"venue":"AAAI","year":2025,"kind":"conference","url":"https://ojs.aaai.org/index.php/AAAI/article/view/33073"},"citation":"@misc{faceantispoofingifas2025,\n  title = {{Interpretable Face Anti-Spoofing: Enhancing Generalization with Multimodal Large Language Models}},\n  author = {Guosheng Zhang and Keyao Wang and Haixiao Yue and Ajian Liu and Gang Zhang and Kun Yao and Errui Ding and Jingdong Wang},\n  year = {2025},\n  eprint = {2501.01720},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2501.01720}\n}\n","citationPath":"/content/papers/citations/face-antispoofing/i-fas.e7abfbdbfc49.bib"},"thumbnail":{"src":"/content/papers/assets/9b282e23d59b1e4c08a08ac3725d9d52ad3fae33f7537e3563d41c260ae095c1.jpg","width":1674,"height":490},"locale":"zh-hant","path":"/content/papers/zh-hant/face-antispoofing/i-fas.9d91b540cd89.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"spoof-aware-captioning-and-filteringscf","text":"Spoof-aware Captioning and Filtering（SCF）","depth":2},{"id":"multimodal-instruction-tuning-with-gac","text":"Multimodal Instruction Tuning with GAC","depth":2},{"id":"globally-aware-connectorgac","text":"Globally Aware Connector（GAC）","depth":3},{"id":"lopsided-language-model-lossl-lm-loss","text":"Lopsided Language Model Loss（L-LM Loss）","depth":2},{"id":"討論","text":"討論","depth":2},{"id":"跨域能力驗證","text":"跨域能力驗證","depth":3},{"id":"模組貢獻","text":"模組貢獻","depth":3},{"id":"caption-類型分析","text":"Caption 類型分析","depth":3},{"id":"l-lm-loss-超參數探討","text":"L-LM Loss 超參數探討","depth":3},{"id":"解釋能力可視化","text":"解釋能力可視化","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"face-antispoofing/faceshield","relation":"comparison","topic":"用多模態語言模型判斷人臉偽造"},{"id":"face-antispoofing/instructflip","relation":"comparison","topic":"用多模態語言模型判斷人臉偽造"}]},{"id":"face-antispoofing/instructflip","category":"face-antispoofing","legacyPath":"face-antispoofing/2507-instructflip","legacySlug":"2507-instructflip","legacyPaths":[],"title":"InstructFLIP","subtitle":"嘆息之壁","author":"Z. Yuan","paperDate":"2025-07-16","publishedOn":"2025-07-18","paperTitle":"InstructFLIP: Exploring Unified Vision-Language Model for Face Anti-spoofing","paperUrl":"https://doi.org/10.1145/3746027.3754939","bibliography":{"title":"InstructFLIP: Exploring Unified Vision-Language Model for Face Anti-spoofing","authors":["Kun-Hsiang Lin","Yu-Wen Tseng","Kang-Yang Huang","Jhih-Ciang Wu","Wen-Huang Cheng"],"url":"https://doi.org/10.1145/3746027.3754939","pdfUrl":"https://arxiv.org/pdf/2507.12060v2","codeUrl":"https://github.com/kunkunlin1221/InstructFLIP","preprint":{"id":"2507.12060","submittedOn":"2025-07-16","url":"https://arxiv.org/abs/2507.12060"},"publication":{"venue":"ACM Multimedia","year":2025,"kind":"conference","url":"https://doi.org/10.1145/3746027.3754939"},"citation":"@misc{faceantispoofinginstructflip2025,\n  title = {{InstructFLIP: Exploring Unified Vision-Language Model for Face Anti-spoofing}},\n  author = {Kun-Hsiang Lin and Yu-Wen Tseng and Kang-Yang Huang and Jhih-Ciang Wu and Wen-Huang Cheng},\n  year = {2025},\n  eprint = {2507.12060},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2507.12060}\n}\n","citationPath":"/content/papers/citations/face-antispoofing/instructflip.d3741ce6514a.bib"},"thumbnail":{"src":"/content/papers/assets/f939a936e3ea5eb6ddd394e6797d1dc1e2b878b9d4750b8826f1baefa55f11c0.jpg","width":1224,"height":664},"locale":"zh-hant","path":"/content/papers/zh-hant/face-antispoofing/instructflip.8eed533d1538.md","headings":[{"id":"定義問題","text":"定義問題","depth":2},{"id":"解決問題","text":"解決問題","depth":2},{"id":"模型架構","text":"模型架構","depth":3},{"id":"內容分支","text":"內容分支","depth":3},{"id":"風格分支","text":"風格分支","depth":3},{"id":"特徵融合與判斷模組","text":"特徵融合與判斷模組","depth":3},{"id":"多重目標學習","text":"多重目標學習","depth":3},{"id":"實作細節","text":"實作細節","depth":3},{"id":"討論","text":"討論","depth":2},{"id":"模組貢獻分析","text":"模組貢獻分析","depth":3},{"id":"llm-模組的影響","text":"LLM 模組的影響","depth":3},{"id":"spoof-類型分類細緻度","text":"Spoof 類型分類細緻度","depth":3},{"id":"style-prompt-多樣性","text":"Style Prompt 多樣性","depth":3},{"id":"質性分析","text":"質性分析","depth":2},{"id":"成功案例分析","text":"成功案例分析","depth":3},{"id":"失敗案例分析","text":"失敗案例分析","depth":3},{"id":"與開源-vlm-模型對比","text":"與開源 VLM 模型對比","depth":3},{"id":"結論","text":"結論","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[{"id":"face-antispoofing/i-fas","relation":"comparison","topic":"用多模態語言模型判斷人臉偽造"},{"id":"face-antispoofing/faceshield","relation":"comparison","topic":"用多模態語言模型判斷人臉偽造"}]}],"text":"還記得我們不久前讀過的那篇資料集的論文 SHIELD 嗎？\n\n這次來看一下後續的相關應用。\n\n## 定義問題\n\n人臉防偽（Face Anti-Spoofing, FAS）是臉部辨識系統中不可或缺的安全機制，防止照片、影片、面具等偽造攻擊誤導模型。過去十年，研究社群已提出大量方法應對這類問題，主流做法大致可分為兩個方向：\n\n第一種是以 CNN 或 Vision Transformer 為骨幹的視覺模型，輔以反射光、深度圖、rPPG、梯度訊號等輔助特徵，將任務視為簡單的活體／假體二元分類；這類方法在訓練資料中表現亮眼，但面對陌生場景與新型攻擊類型時，模型泛化能力明顯不足。\n\n第二種是近期興起的 Vision-Language 方法，改用圖像與文字配對訓練模型，以 CLIP 為代表。這些方法透過語言描述的語意訊號輔助判斷，改善了傳統模型對 spurious features 的依賴。\n\n不過 VLM 的模型仍有兩大問題未解：\n\n- 其一，缺乏「解釋力」，難以回答為何一張臉是假的；\n- 其二，無法定位假臉中遭偽造的區域，也無法同時處理分類與推理任務。\n\n更麻煩的是，無論是視覺模型還是 VLM 的方法，其訓練資料都高度貧乏。多數 FAS 資料集僅包含標籤與圖像，缺乏能支援推理任務的敘述式註解，這直接限制了模型的語言能力與泛化潛力。\n\n那麼，Multimodal LLM（MLLM）呢？\n\n儘管 GPT-4V、LLaVA 等通用 MLLM 已在遙測、醫療、Deepfake 偵測等任務上展現強大能力，但在面對 FAS 任務時，這些模型卻表現平平。主要原因在於：目前語言模型本身缺乏針對假臉攻擊的知識，而現成的 vision encoder 難以察覺真假臉間微妙的紋理落差。\n\n曾有研究試圖用指令微調（instruction tuning）打造具備描述能力的 FAS 模型，並整合多個資料集以提升語意辨識，但這類方法仍無法同時處理分類、推理與區域定位三項任務。而現實世界中的攻擊往往是複合型態，這些限制讓現有模型難以在實務中站穩腳步。\n\n於是問題來了：\n\n> **我們能不能打造一種模型，真正看懂一張假臉的可疑之處、說得出為什麼，甚至指出它是從哪裡開始欺騙的？**\n\n## 解決問題\n\n要讓一個多模態語言模型學會識破假臉，光靠文字和圖片還不夠。\n\n它需要任務明確、語意完整、結構合理的訓練資料，甚至需要從不同層次去理解「這張臉為什麼可疑」。而作者認為關鍵的起點，就是資料。\n\n為此，作者首先規劃了資料集的設計概念，如下圖：\n\n![dataset](/content/papers/assets/8a1bf53dfe4496b9a74be55f8eb34afa3fbf312459ba47dd8b7f245c351c2686.jpg)\n\nFaceShield 的訓練資料分為兩個階段構成：\n\n- **FaceShield-pre10K**：用於預訓練，讓模型具備初步的描述與理解能力。\n- **FaceShield-sft45K**：用於指令微調（SFT），強化模型在多任務上的具體表現。\n\n研究團隊整合了 WMCA、PADISI、SiW-Mv2 等常用 FAS 資料集，將各自的分類統一為 12 種類別，包括：\n\n> Bonafide、Fakehead、Print、Glasses、Replay、Paper mask、Flexible mask、Rigid mask、Partial Eye、Partial Mouth、Makeup、Tattoo\n\n其中不只保留圖像層級的標註，還針對局部偽造區域進行精細的框選與座標標記，最終整理出 12,091 張帶有類別標籤的圖像，以及 3,139 張具備偽造區域框選資訊的圖像。\n\n圖像與標籤蒐集完畢後，接下來要讓語言模型理解這些資訊。\n\n作者使用 Bunny-Llama-3-8B-V 作為生成助理，透過系統提示語（system prompt）引導模型輸出描述性語句，並依據不同任務類型生成對應的問答資料。\n\n對於 **FaceShield-pre10K**，流程較為單純：模型僅根據圖像產生敘述文字，配對後經過 CLIP 相似度篩選（低於 15% 則剔除），以濾除相似度較低的圖文配對。\n\n而 **FaceShield-sft45K** 則更為嚴謹，涵蓋四種任務指令，並搭配多層次的過濾與增強策略：\n\n- MLLM 先輸出多種 QA 配對，經人工與關鍵字濾除錯誤項目\n- 接著使用 LLaMA3 對保留下的 QA 種子資料進行語言增強，提升語句多樣性與對話流暢性\n\n四種任務設計如下圖所示：\n\n![example](/content/papers/assets/570d619ed2b90e46a9dc5c3bbbf2b9276c5500d76be20e951bf49e8853e84590.jpg)\n\n- **粗分類（Coarse-grained Classification）**：辨識該臉為真或假\n- **細分類（Fine-grained Classification）**：辨識假臉的具體攻擊類型\n- **推理判斷（Reasoning）**：從圖像特徵出發，文字化描述判斷依據\n- **區域定位（Attack Localization）**：偵測假臉區域，標出攻擊位置的座標\n\n### 模型架構\n\n![model arch](/content/papers/assets/72227f25d8bae3bb37fab6ee6e697210da60a8f59ddb17f11dd2634a8bb762df.jpg)\n\n訓練一個針對人臉防偽（FAS）任務的多模態大型語言模型（MLLM），有兩個核心目標：\n\n1. **讓視覺編碼器更擅長分辨真臉與偽臉之間細微差異；**\n2. **讓語言模型能依語意提示回答不同任務的問題。**\n\n然而，若直接以 RGB 影像配合 QA 配對進行訓練，如上圖 (a) 所示，模型將面臨一個挑戰：\n\n- **RGB 空間中，真臉與假臉太像了**。\n\n這讓模型難以穩定學習到可靠的區辨特徵，泛化能力也因此受限。\n\n為了解決這個問題，作者導入了兩個關鍵模組：\n\n- **SAVP（Spoof-Aware Vision Perception）**：從視覺前處理出發，加強模型對假臉細節的感知；\n- **PVTM（Prompt-Guided Vision Token Masking）**：透過語言提示動態調整模型的注意力焦點。\n\n這兩個模組共同建立了一個更敏感、更解釋性更強的 FAS MLLM。\n\n### Spoof-Aware Vision Perception（SAVP）\n\n如上圖 (b) 所示，SAVP 的核心思路是：**RGB 空間中的外觀不足以有效區分活體與攻擊，但結合影像紋理與梯度資訊，就可能放大這些細微差異。**\n\n具體作法如下：針對每張輸入影像，SAVP 額外計算三種局部特徵，並將它們組成三通道：\n\n- **LBP（Local Binary Pattern）**：反映局部紋理變化\n- **Gray（灰階）**：去除色彩干擾，強化亮度結構\n- **HOG（Histogram of Oriented Gradients）**：捕捉邊緣方向與梯度能量分布\n\n這些影像特徵以三通道形式串接後輸入視覺編碼器，提取出 spoof-aware 的特徵表示 $V_{\\text{SAV}}$，並與原始 RGB 特徵 $V_{\\text{RGB}}$ 串接為總視覺輸入：\n\n$$\nV_{\\text{SAV}} = \\text{Encoder}([\\text{LBP}, \\text{Gray}, \\text{HOG}])\n$$\n\n$$\nV = [V_{\\text{RGB}}, V_{\\text{SAV}}]\n$$\n\n此設計使模型獲得兩種視覺認知：一種來自外觀，一種來自紋理與結構。\n\n### Prompt-Guided Vision Token Masking（PVTM）\n\n\n\n![pvtm](/content/papers/assets/2a85639c45b2d561385f49ac882cf5ea84b00201692426f8205e3f98b78249ec.jpg)\n\n\n\n即便提取了豐富的視覺特徵，如果模型無法聚焦於「任務相關的區域」，則仍可能被干擾。\n\n為此，PVTM 提供一種動態選擇視覺 token 的機制，方法如下：\n\n- 將語言提示的文字 token 池化為 $P$，再計算每個視覺 token $V_i$ 與 $P$ 的餘弦相似度：\n\n  $$\n  \\text{Sim}(V_i, P) = \\frac{V_i \\cdot P}{\\|V_i\\| \\cdot \\|P\\|}\n  $$\n\n- 再以 softmax 轉換為權重排序分數：\n\n  $$\n  S_{\\text{rank}}(V_i, P) = \\frac{e^{\\text{Sim}(V_i, P)}}{\\sum_j e^{\\text{Sim}(V_j, P)}}\n  $$\n\n接著保留前 k% 的高重要性 token，並對剩餘的部分以 p% 機率隨機遮蔽；實驗設定為 k=10%、p=5%。\n\n這使得模型對於「分類」、「推理」、「區域定位」等任務的視覺焦點會因 prompt 而異，增強任務適應性與抗干擾能力。\n\n### 模型流程\n\n\n\n![model flow](/content/papers/assets/1efb2791352953106e589acfc4adf3ae6d0e5d722946ba7cbece5fb75b8513a7.jpg)\n\n\n\n整體架構如上圖 (c) 所示，FaceShield 的資訊流動如下：\n\n1. 將輸入影像轉換為 $V_{\\text{RGB}}$ 與 $V_{\\text{SAV}}$ 兩組特徵；\n2. 合併後經過 PVTM 選取最重要視覺 token；\n3. 將選中的 token 投影並與語言 prompt 對齊，得到融合輸入 $V_{\\text{align}}$：\n\n   $$\n   V_{\\text{align}} = \\text{Projection}(V_{\\text{RGB}}, V_{\\text{SAV}})\n   $$\n\n4. 將 $V_{\\text{align}}$ 和 prompt 一起輸入語言模型，輸出任務預測結果 $Y$：\n\n   $$\n   Y = \\text{MLLM}(V_{\\text{align}}, P)\n   $$\n\n### 二階段訓練\n\nFaceShield 採用兩階段訓練策略：\n\n- **第一階段：預訓練（Pretraining）**\n\n  在這個階段，FaceShield 使用的是前面建立的 **FaceShield-pre10K**，目標是讓視覺 encoder 所提取的特徵能與語言提示對齊，**建立視覺與語言之間的通道對應關係**。\n\n  - 所使用的 vision encoder 是來自預訓練模型，僅微調其後接的 **Projector 與 PVTM**。\n  - 訓練策略為 **continual pretraining**，即在原有權重基礎上繼續訓練；\n  - 損失函數為 **cross-entropy loss**，以 next-token prediction 作為語言目標；\n  - 預訓練僅進行 **一個 epoch**，重點在於快速對齊嵌入空間，而非強化生成語意。\n\n- **第二階段：指令微調（Supervised Fine-tuning, SFT）**\n\n  接下來是關鍵的任務適應階段，FaceShield 使用 **FaceShield-sft45K** 對模型進行指令微調，使其學會回答四種特定任務問題。\n\n  微調對象包括：\n\n  - **LoRA 層（語言模型內部輕量參數）**\n  - **Projector（視覺與語言橋接模組）**\n\n  使用 LoRA（Low-Rank Adaptation）對大型語言模型進行高效調整，每個 Transformer block 皆插入一組 LoRA 子模組，具體參數如下：\n\n  - Hidden size：128\n  - Scaling factor：256\n  - 損失函數與預訓練階段相同，採用 cross-entropy loss 進行 next-token prediction，但調整重點轉向語言表現與任務回應。\n\n## 討論\n\n### Intra-dataset 評估\n\n\n\n![intra-dataset benchmark](/content/papers/assets/8bca2a1b1f7326a820347596458a0b27dfc24e1a7bd8f37a42255964ece014ab.jpg)\n\n\n\n實驗結果顯示，FaceShield 在三個來源資料集（W/S/P）的合併測試集上，HTER 低於表中的傳統 FAS 方法與通用 MLLM：\n\n1. **傳統 FAS 方法**\n\n   - 表中列出 ResNet、PatchNet 與 CoOp，三者的架構和訓練方式不同。\n   - 表中三個方法的 HTER 為 1.27%～2.32%，高於 FaceShield 的 0.53%。\n\n2. **通用型 MLLM 的 zero-shot 能力**\n\n   - 此類模型未經任何針對 FAS 任務的調整，僅靠先前語言／視覺知識進行推斷。\n   - 表中的 LLaVA、Qwen-VL、MiniGPT4 和 Bunny，HTER 均高於 FaceShield；這裡沒有比較 GPT-4V。\n\n3. **自行微調的對照組 MLLM：Bunny**\n\n   - 作者進一步將開源 MLLM「**Bunny**」用相同資料進行微調（SFT），作為嚴格對照。\n   - 結果顯示：**FaceShield 的 HTER 為 0.53%，比微調後 Bunny 的 1.52% 低 0.99 個百分點**。\n\n### Cross-dataset 評估\n\n\n\n![cross-dataset benchmark](/content/papers/assets/9d6e462e896b969fb13fec85c16f888c516a7a245528362738c1e462212f0486.jpg)\n\n\n\n上表呈現的是跨資料集泛化能力評估結果。\n\n每次訓練選用三個資料集中的兩個（例如 S + P），第三個（如 W）完全不見於訓練，作為測試集。測試流程記作 `S&P → W`，共三種排列組合，對應三個 domain。\n\n結果顯示在三種設定中，FaceShield 的 HTER 均低於表中的傳統方法；`S&P → W` 協議下，表列 HTER 為 5.71%。這張表沒有 MLLM 對照組，也不足以判定該協議是三者中最具挑戰的一組。\n\n### 消融實驗\n\n1. **指令資料集有效性**\n\n   如果指令資料有效，那能不能讓其他模型也變強？\n\n   實驗方法是將同樣資料集套用在 LLaVA、Bunny 等開源 MLLM 上進行預訓練與微調。\n\n   結果如下圖所示：\n\n\n\n    ![prompt](/content/papers/assets/1b96d53f54730942cf2096a14082ee37e00f686febe06eed2b996f46bfd02af4.jpg)\n\n\n\n   兩個模型在圖中的粗分類、細分類與推理結果均有提升；例如 LLaVA 的粗分類 HTER 從 27.76% 降至 1.74%，Bunny 則從 17.87% 降至 1.06%。圖中未報告跨任務的平均提升幅度。\n\n   這支持資料集對這兩個受測 MLLM 有幫助。\n\n2. **SAVP（Spoof-Aware Vision Perception）模組有效？**\n\n   ![savp](/content/papers/assets/f5e3955ddaa2403f0350372dbc929b8211d32e600b208ee78868bbe086829094.jpg)\n\n   比較上表前兩列，第一列為 baseline（僅用 RGB），第二列加入 SAVP。\n\n   實驗結果顯示，在四項任務中均有提升；攻擊區域定位的 AP@40 從 92.30% 升至 97.09%，AP@50 從 89.71% 升至 95.21%。\n\n   這組消融支持加入局部紋理與梯度資訊有助於此設定下的辨識與區域定位。\n\n3. **PVTM（Prompt-Guided Vision Token Masking）模組**\n\n   同樣參考剛才那張表格的最後一列。\n\n   相較於只加入 SAVP，PVTM 使粗分類 HTER 從 1.06% 降至 0.53%、細分類 ACC 從 94.59% 升至 95.81%，區域定位 AP@40／AP@50 也上升；**推理任務則小幅下降**（ACC 99.41%→99.29%、HTER 0.48%→0.57%）。作者推測遮蔽可能使推理所需資訊流失，消融實驗未單獨驗證這個原因。\n\n### 可視化分析\n\n\n\n![vis](/content/papers/assets/e602849f6b70c6dd413cebc15ec2c19750fc9a75b50caa9af4cbd219a305f156.jpg)\n\n\n\n如果一個模型說這張臉是假的，那麼它是看著「哪裡」說出這句話的？\n\n作者試著給我們提供這個答案，上圖 (a) 展示了視覺 token 對語言 prompt 的回應強度（token importance），涵蓋三種不同類別的圖片。\n\n圖中較高的 token 重要性多落在臉部區域，但也可見其他位置的回應；這張相似度視覺化無法單獨證明哪些特徵決定了最終輸出。\n\n這些例子顯示視覺 token 對不同任務提示的回應分布。\n\n接著上圖 (b) 對比了 RGB 與 SAV token 在「偽裝區域偵測」任務中的表現差異。\n\n- 左圖為一個戴眼部假面（eye mask）的樣本：\n\n  - RGB token 的重要性分布**非常分散**，模型缺乏明確指向；\n  - SAV token 的高重要性區域較集中於**眼部攻擊區域**；\n\n- 右圖則顯示眼部攻擊區域與持物的手：\n\n  - RGB 與 SAV token 的重要性分布在這些區域提供互補線索；圖中沒有單獨量測手部遮擋的偵測準確率。\n\n## 結論\n\n在這篇論文中，作者將傳統的二元辨識任務擴展為包含粗分類、細分類、語意推理與區域定位的多任務架構。模型不只指出這張臉是真是假，更能說明它為什麼這麼判斷、哪一個區域出現了異常，甚至在語言的引導下聚焦於最具判斷價值的圖像特徵。\n\n這樣的能力，讓防偽系統得以從單純的分類器，轉變為一個具備語意邏輯、可視解釋與多步推理能力的判別者，而不會只停留在給出「是或否」的答案。\n\n說出「為什麼是假的」、「假在哪裡」的能力，是現在我們對模型的期待，也是主要的研究潮流。\n\n\n> [!TIP]\n> \n> 之前我們看過了 I-FAS。\n> \n> I-FAS 主要貢獻在於證明敘述可以成為監督信號，而 FaceShield 則是補齊視覺細節，讓語言、紋理與空間位置都能同時進入決策回路。\n> \n> 若讀者偏好簡潔的模型與精緻的 caption 設計，I-FAS 確實更「純粹」；但如果目標是面向全天候、跨設備且需反覆說服終端用戶的實際系統，FaceShield 在任務覆蓋與定位解釋上的完整度，會是更完整的設計。\n","imageSizes":{"/content/papers/assets/8a1bf53dfe4496b9a74be55f8eb34afa3fbf312459ba47dd8b7f245c351c2686.jpg":{"width":1224,"height":562},"/content/papers/assets/570d619ed2b90e46a9dc5c3bbbf2b9276c5500d76be20e951bf49e8853e84590.jpg":{"width":1826,"height":870},"/content/papers/assets/72227f25d8bae3bb37fab6ee6e697210da60a8f59ddb17f11dd2634a8bb762df.jpg":{"width":1436,"height":560},"/content/papers/assets/2a85639c45b2d561385f49ac882cf5ea84b00201692426f8205e3f98b78249ec.jpg":{"width":1224,"height":908},"/content/papers/assets/1efb2791352953106e589acfc4adf3ae6d0e5d722946ba7cbece5fb75b8513a7.jpg":{"width":550,"height":560},"/content/papers/assets/8bca2a1b1f7326a820347596458a0b27dfc24e1a7bd8f37a42255964ece014ab.jpg":{"width":952,"height":628},"/content/papers/assets/9d6e462e896b969fb13fec85c16f888c516a7a245528362738c1e462212f0486.jpg":{"width":868,"height":824},"/content/papers/assets/1b96d53f54730942cf2096a14082ee37e00f686febe06eed2b996f46bfd02af4.jpg":{"width":1224,"height":604},"/content/papers/assets/f5e3955ddaa2403f0350372dbc929b8211d32e600b208ee78868bbe086829094.jpg":{"width":1746,"height":216},"/content/papers/assets/e602849f6b70c6dd413cebc15ec2c19750fc9a75b50caa9af4cbd219a305f156.jpg":{"width":1008,"height":776}}}
