by Holyn Kanake, @kanakhey
The Persistence of Memory, Salvador Dalí, 1931.
I distinctly remember the first time I saw Salvador Dalí's The Persistence of Memory. I was caught in this imaginary universe—a true contemplation of the relativity of space and time. Dalí expertly created an extended reality, a dream-state that seemed almost impossible ... until now.
Synthetic media will create an extended reality in the next decade. One that will deploy technology in new ways, transform consumer behaviors, and introduce new business models.
Synthetic media is algorithmically created or modified video, images, audio, or combined media . The pernicious forms of this media are eloquently called deepfakes.
Technically speaking, synthetic media leverages generative adversarial networks (GANs). In this deep learning, unsupervised machine learning technique, there are two networks essentially battling one another to create new or altered media (data). One generates the synthetic media while the other tries to discriminate the authenticity of the media. To generate the most realistic media, GANs must learn to generate any distribution of data in an undetectable manner. This framework was proposed by Ian Goodfellow and other researchers in 2014.
Since 2014, researchers and engineers have created synthetic media in the form of audio, video, and images. The last year has seen the rise of media that synthesizes both video and audio, images and audio, and even video and text. It still takes technical skill to produce synthetic media and the output is often imperfect; however, the technology is moving quickly and soon real media will be indistinguishable from synthetically produced media.
This rapid progress recently detailed by Benjamin Goggin. Synthetic media has come far from it's mainstream origins on Reddit in 2017. Research is moving even faster, but government and tech policies are still catching up to the impact this emerging technology.
June 2019: UC Berkeley detects deepfakes through a person's unique facial expressions and movements with a 92% success rate (URL)
June 2019: Adobe Research and UC Berkeley detected facial manipulations in Adobe Photoshop (URL)
June 2019: Stanford, Max Planck Institute, Princeton, and Adobe Research created an algorithm that edits video to change the speech content or to remove filler words (URL)
May 2019: MIT CSAIL reconstructed a facial image of a person from a short audio recording of that person speaking (URL)
May 2019: Samsung AI created a system that meta-learns on a large dataset of videos and then generated talking head models from one data point of a previously unseen subject (URL)
Few-Shot Adversarial Learning of Realistic Neural Talking Head Models, Samsung AI, May 2019.
March 2019: Fernando De Meer (QuantDare) generated synthetic financial data to use in trading model development (URL)
<aside> 📜 There have been four state bills introduced in the US to combat synthetic media in California, Texas, New York, and Massachusetts. The bill in California failed to pass while others are still pending. Those against these bills argue that they infringe on civil rights, particularly free speech.
</aside>