Abstract

In the last two decades, we have witnessed an immense increase in the use of multimedia content on the internet, for multiple applications ranging from the most innocuous to very critical ones. Naturally, this emergence has given rise to many types of threats posed when this content can be manipulated/used for malicious purposes. For example, fake media can be used to drive personal opinions, ruining the image of a public figure, or for criminal activities such as terrorist propaganda and cyberbullying. The research community has of course moved to counter attack these threats by designing manipulation-detection systems based on a variety of techniques, such as signal processing, statistics, and machine learning. This research and practice activity has given rise to the field of multimedia forensics.

The success of deep learning in the last decade has led to its use in multimedia forensics as well. In this survey, we look at the latest trends and deep-learning-based techniques introduced to solve three main questions investigated in the field of multimedia forensics. We begin by examining the manipulations of images and videos produced with editing tools, reporting the deep-learning approaches adopted to counter these attacks. Next, we move on to the issue of the source camera model and device identification, as well as the more recent problem of monitoring image and video sharing on social media. Finally, we look at the most recent challenge that has emerged in recent years: recognizing deepfakes, which we use to describe any content generated using artificial-intelligence techniques; we present the methods that have been introduced to show the existence of traces left in deepfake content and to detect them. For each problem, we also report the most popular metrics and datasets used today.

Deep Learning for Multimedia Forensics

Over the past few decades, we have witnessed a huge increase in the use of multimedia content on the internet, for multiple applications ranging from innocuous to critical ones. This development has led to threats posed when content can be manipulated/used for malicious purposes. For example, fake media can be used to drive personal opinions, or for criminal activities such as terrorist propaganda and cyberbullying. This research and practice activity gave rise to the creation of the multimedia forensics field.

In this monograph, the latest trends and deep-learning-based techniques for multimedia forensics are introduced, in both architectural and data-processing. Firstly, different techniques used to manipulate content are presented, followed by image and video forgery techniques. Thereafter, deep learning methods for source identification and recent solutions for deepfake detection are covered. Datasets and evaluation metrics are included, and conclusions are presented.

The publication is intended for researchers, students and professionals active in the fields of Deep Learning and Multimedia Forensics.