aiqbench analysis
This research introduces DF-MoE, a multimodal sparse Mixture-of-Experts framework designed to improve the generalization of deepfake detectors across diverse generation methods. By integrating high-level cues from audio and visual modalities using pre-trained models, the system achieves superior performance on five benchmarks compared to state-of-the-art techniques. The work addresses a critical challenge in AI safety by offering a robust solution for detecting synthetic media that maintains effectiveness when facing evolving deepfake tactics.
DF-MoE: Generalizable Deepfake Detection via Multimodal Sparse Mixture-of-Experts
arXiv:2608.23363v1 Announce Type: cross Abstract: Audio-visual deepfake detection is an actively studied topic, where one of the main challenges is to develop detectors able to generalize across deepfake generation methods. We conjecture that overfitting can be mitigated by extracting multiple high-level cues from the available audio and visual modalities via pre-trained models. We therefore assemble a wide variety of pre-trained models to extract features that encode mouth movements, face parsing, facial expressions, head pose, gaze tracking, heart rate, audio emotion and speech activity. We further integrate both unimodal and multimodal cues via a Mixture-of-Experts (MoE) backbone to detect deepfakes. We perform in-domain and cross-domain experiments on five benchmarks for deepfake detection (MAVOS-DD, AVLips, PolyGlotFake, BioDeepAV, FakeAVCeleb) to compare our framework (DF-MoE) with state-of-the-art methods. Our results indicate that DF-MoE obtains superior deepfake detection results, surpassing all competing methods. We release our code at https://github.com/vladhondru25/DF-MoE.
Abstract:Audio-visual deepfake detection is an actively studied topic, where one of the main challenges is to develop detectors able to generalize across deepfake generation methods. We conjecture that overfitting can be mitigated by extracting multiple high-level cues from the available audio and visual modalities via pre-trained models. We therefore assemble a wide variety of pre-trained models to extract features that encode mouth movements, face parsing, facial expressions, head pose, gaze tracking, heart rate, audio emotion and speech activity. We further integrate both unimodal and multimodal cues via a Mixture-of-Experts (MoE) backbone to detect deepfakes. We perform in-domain and cross-domain experiments on five benchmarks for deepfake detection (MAVOS-DD, AVLips, PolyGlotFake, BioDeepAV, FakeAVCeleb) to compare our framework (DF-MoE) with state-of-the-art methods. Our results indicate that DF-MoE obtains superior deepfake detection results, surpassing all competing methods. We release our code at this https URL.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2608.23363 [cs.CV]
(or arXiv:2608.23363v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.23363 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 244 words
Related coverage
[AINews] OpenAI reports Navier-Stokes singularity find in 88 hours using Astra-next, roughly 10,000 agents and 130B tokens (>$40M), a contender for second ever Millennium Prize awarded
Latent Space · Sep 9, 2026
Efficient MiniMax-M3 Inference on AMD Instinct GPUs with ATOM and ATOMesh
AMD · Jul 21, 2026
Links #5: 2026/07
LessWrong · Jul 31, 2026
Promptimus: Improving already good LLM prompts with zero manual engineering
Amazon Science · May 14, 2026