Full-defense framework: multi-level deepfake detection and source tracing

Abstract

Deepfake poses significant threats to various fields, including politics, journalism, and entertainment. Although many defense methods against deepfake have been proposed based on either passive detection or proactive defense, few have achieved both passive detection and proactive defense. To address this issue, we propose a full-defense framework (FDF) based on cross-domain feature fusion and separable watermarks (SepMark) to achieve copyright protection and deepfake detection, combining the ideas of passive detection and proactive defense. The proactive defense module consists of one encoder and two separable decoders, where the encoder embeds one watermark into the protected face, and two decoders separately extract two watermarks with different robustness. The robust watermark can reliably trace the trusted marked face while the semi-robust watermark is sensitive to malicious distortions that make the watermark disappear after deepfake or watermark removal attack. The passive detection module fuses spatial- and frequency-domain features to further differentiate between deepfake content and watermark removal attacks in the absence of watermarks. The proposed cross-domain feature fusion involves substituting the “secondary” channels of spatial-domain features with the “primary” channels of frequency-domain features. Subsequently, the “primary” channels of spatial-domain features are used to replace the “secondary” channels of frequency-domain features. Extensive experiments demonstrate that our approach not only offers proactive defense mechanisms by using extracted watermarks, i.e., source tracing and copyright protection, but also achieves passive detection when there are no watermarks, to further differentiate between deepfake content and watermark removal attacks, thereby offering a full-defense approach.

1. Introduction

The rapid advancement of deepfake technology poses significant threats to individual privacy and societal trust. As synthetic media generation evolves, the risk of misinformation increases, raising concerns about digital content integrity and media credibility. This evolution threatens fields like journalism and privacy protection.

In response, research on deepfake defense has grown, focusing on passive detection and proactive defense techniques. Passive detection relies mainly on convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Several approaches have been proposed, including two-stream CNNs to capture tampering artifacts and local noise residuals, CNNs combined with visual Transformers, Transformer-based self-supervised learning, and spatial-temporal contrastive learning. However, passive detection methods often struggle with post-processed forgeries and compressed social media content, motivating the need for integrated defense strategies that combine passive and proactive mechanisms.