Synthetic Media, Deepfakes, and the Crisis of Evidence

For approximately 150 years

For approximately 150 years — from the invention of photography in the 1830s to the development of generative AI in the 2010s — the photograph served as a special kind of evidence. A photograph was not a statement, which could be true or false. It was a mechanical record, produced by the interaction of light with a photosensitive surface, and it carried, by virtue of its mechanical nature, a kind of authority that human testimony did not. The camera, as the saying went, did not lie. The saying was never entirely true — photographs could be staged, cropped, and manipulated — but it was true enough, for long enough, that the photograph became one of the foundational instruments of modern evidence. Courts admitted photographs. Newspapers published them. People believed them. That era is ending. The development of synthetic media

— AI-generated images, videos, and audio that are, by most measures, indistinguishable from real recordings — is collapsing the authority of the photographic record, and it is launching an arms race between the technologies of generation and the technologies of detection that will, in all likelihood, define the information environment of the twenty-first century. This is the

Synthetic Media / Deepfakes

AI-generated audio, video, or images that depict real people doing or saying things they did not — produced by GANs and later diffusion models. The term “deepfake” originated in 2017 from a Reddit user; synthetic media now includes voice clones, face swaps, and entirely synthetic personas.

The Deepfake

The word “deepfake” was coined in late 2017 (December 2017, when Vice Motherboard’s Samantha Cole broke the story) by a Reddit user who called themselves “deepfakes” — eventually identified as a programmer using the handle “deepfakeapps.” The user posted a series of pornographic videos in which the faces of real actresses — Scarlett Johansson, Gal Gadot, Taylor Swift, and others — had been digitally superimposed, using a machine-learning technique, onto the bodies of the adult-film performers. The videos were, by the standards of 2017, remarkably convincing — the face swaps were smooth, the expressions were natural, and the result was, to a casual viewer, difficult to distinguish from real footage. The user had used a combination of open-source machine-learning tools, including Google’s TensorFlow (Google’s open-source ML framework), to train a model that could swap faces in video.

The videos were, by most accounts, the first widely-seen examples of what would come to be called “deepfakes” — synthetic media produced by deep learning. The term itself was a portmanteau of “deep learning” and “fake,” and it quickly entered the standard vocabulary. Within months of the original Reddit posts, the technology had been packaged into consumer-friendly tools. FakeApp, released in January 2018, was the first widely available deepfake application — a desktop program that allowed anyone with a sufficiently powerful graphics card to create face-swap videos. The tool was, by most accounts, simple enough that a non-technical user could produce a convincing deepfake in a few hours.

The democratization of deepfake technology was, for many observers, alarming. The technology had obvious applications in disinformation, in political manipulation, in harassment, and in the production of non-consensual pornography. The original Reddit deepfakes were, themselves, a form of non-consensual pornography — the faces of real women, used without their consent, in explicit videos. The concern was that, as the technology improved and spread, the range of harmful applications would grow, and the ability to counter them would diminish.

The Obama PSA and the Mainstream Moment

The deepfake phenomenon entered the mainstream on April 17, 2018, when the comedian Jordan Peele and the BuzzFeed CEO Jonah Peretti produced a public service announcement using deepfake technology. The PSA showed what appeared to be Barack Obama delivering a speech — but it was, in fact, a deepfake, with Peele’s voice and mannerisms digitally mapped onto Obama’s face. The video was, by most accounts, convincing — a casual viewer would have had difficulty distinguishing it from real footage of Obama. The PSA ended with Peele, in his own voice, warning about the dangers of deepfakes: “This is a dangerous time. Moving forward, we need to be more vigilant with what we trust from the internet.”

The PSA was, by most measures, a watershed moment. It was covered by every major news outlet. It was viewed millions of times. And it established, in the public consciousness, the concept of the deepfake — the idea that AI could produce video that was indistinguishable from reality, and that this capability posed a threat to the information environment. The PSA was, in some ways, a public service — it warned people about the technology. But it was also, in some ways, a demonstration — it showed people what the technology could do, and it raised the question of what would happen when the technology fell into less responsible hands.

Voice Cloning

The deepfake phenomenon was not limited to video. Voice cloning — the AI-based synthesis of human speech — developed in parallel, and it presented its own set of challenges. The technology, which used machine-learning models to analyse a person’s voice and to generate new speech that sounded like that person, improved dramatically in the late 2010s and early 2020s. By 2022, voice cloning had become accessible to consumers, through companies like ElevenLabs (founded in 2022) and through open-source tools that allowed anyone with a modest amount of computing power to clone a voice from a few minutes of audio.

The voice-cloning technology was, in many ways, more dangerous than video deepfakes. A video deepfake required, at minimum, a convincing visual synthesis — a task that, while increasingly feasible, still required significant computing power and technical skill. A voice clone, by contrast, could be produced from a short audio sample, using consumer-grade tools, in a matter of minutes. The resulting audio could be used, over the phone or in a voice message, to impersonate a real person — a CEO, a family member, a government official — in ways that were extremely difficult to detect.

The most prominent example of voice-cloning abuse came in January 2024, when a political operative named Steven Kramer used an AI-generated voice clone of President Joe Biden to produce a robocall that was sent to thousands of voters in New Hampshire, ahead of the state’s primary election. The robocall, which used a convincing imitation of Biden’s voice, urged voters not to participate in the primary — “your vote makes a difference in November, not on Tuesday.” The robocall was, by most accounts, a deliberate attempt at voter suppression, and it triggered a regulatory response. The Federal Communications Commission proposed a $6 million fine against Kramer (Notice of Apparent Liability issued February 22, 2024), and the carrier that transmitted the robocall, Lingo Telecom, was fined $1 million. The incident was, for many observers, a preview of the role that synthetic media would play in the 2024 election (see B84).

The Liar’s Dividend

The most insidious consequence of synthetic media is not, however, the deepfake itself. It is what law professors Robert Chesney (of the University of Texas) and Danielle Citron (of the University of Virginia, later Boston University) called the “liar’s dividend.” Chesney and Citron introduced the concept in their 2019 paper, “Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security,” published in the California Law Review (107 Calif. L. Rev. 1753). The liar’s dividend is the phenomenon in which the very existence of deepfakes — the knowledge that audio and video can be fabricated — makes it possible to dismiss real evidence as fake.

The logic is straightforward. Before deepfakes, if a video showed a politician saying something embarrassing, the politician had limited options: they could deny saying it (but the video was evidence), they could apologise, or they could try to change the subject. After deepfakes, the politician has a new option: they can claim that the video is a deepfake. The claim does not need to be true — it just needs to be plausible enough to create doubt. And the existence of deepfake technology makes the claim plausible, because everyone knows that deepfakes exist and that they are convincing.

The liar’s dividend is, in some ways, more dangerous than the deepfake itself. A deepfake is a specific piece of false content, which can, in principle, be detected and debunked. The liar’s dividend is a general erosion of trust in all evidence — a collapse of the shared assumption that audio and video recordings can be trusted. Once that assumption collapses, the evidence base for journalism, for law enforcement, for historical record, and for democratic accountability is fundamentally undermined. Every piece of evidence becomes contestable, and the contest is not about what the evidence shows but about whether the evidence is real.

The liar’s dividend has, by the mid-2020s, become a standard part of the political landscape. Politicians, confronted with damaging recordings, routinely suggest that the recordings might be fake — and the suggestion, however baseless, finds traction in an environment where the existence of deepfakes has made all recordings suspect. The phenomenon is, for democracy, a serious threat. A political system that depends on accountability — on the ability of the press to report, of the public to judge, and of the legal system to adjudicate on the basis of evidence — cannot function if the evidence itself is universally distrusted.

The Detection Arms Race

The response to the deepfake threat has been, in large part, a technological arms race. On one side are the generators — the AI systems that produce synthetic media, which are improving rapidly, driven by the same advances in machine learning that are driving the broader AI revolution. On the other side are the detectors — the systems that attempt to identify synthetic media, distinguishing it from real recordings. The detection research community, led by researchers like Hany Farid (at UC Berkeley) and Siwei Lyu (at the University at Buffalo, who built the DeepFake-o-Meter, a public deepfake detection tool), has produced a range of detection techniques, from pixel-level analysis (detecting AI-image artefacts at the pixel level — a technique that loses effectiveness as generators improve) to linguistic analysis that identifies patterns in AI-generated text.

The arms race has, by most accounts, not been going well for the detectors. The generative models are improving faster than the detection models, and each advance in generation makes the previous generation of detectors obsolete. The problem is fundamental: a detector that is trained to identify the artefacts of a particular generation model will fail when a new model, with different artefacts, is released. The generator has the advantage — it only needs to produce convincing output, while the detector needs to identify every possible kind of fake. Hany Farid, in a June 2026 New York Times interview, put it bluntly: “the technology to create fake videos and images is improving too quickly to keep up.”

The detection problem is compounded by the open-weights movement (see B78). When generative models are freely available, anyone can use them, modify them, and build on them — and the resulting models produce outputs that the standard detectors have not been trained to identify. The open-weights approach, which has democratised generative AI, has also democratised the production of synthetic media, and the detection community has struggled to keep pace.

Content Provenance and the C2PA

The failure of detection has led, in recent years, to a shift in strategy. Rather than trying to detect fakes after the fact, a growing community of researchers, companies, and policymakers has focused on provenance — on establishing, at the point of creation, that a piece of media is real, and on providing a cryptographic chain of custody that allows that claim to be verified.

The most prominent provenance initiative is the C2PA — the Coalition for Content Provenance and Authenticity — founded on February 22, 2021, by Adobe (through its Content Authenticity Initiative), Microsoft, the BBC, Intel, Truepic, and Arm. (The C2PA merged Adobe’s Content Authenticity Initiative with the BBC/Microsoft Project Origin.)

The coalition developed a technical standard for attaching provenance information — information about who created a piece of media, when, and with what tools — to the media itself, in a way that is cryptographically secure and that can be verified by anyone who receives the media. The standard, which has been adopted by a range of camera manufacturers, software companies, and media organisations, is an attempt to restore the authority of the photographic record — not by detecting fakes, but by authenticating reals.

The C2PA approach has, however, its own limitations. The provenance information is only useful if it is present — if the creator of the media has chosen to attach it. For media created by professional photographers, news organisations, and other institutional actors, this is feasible. But for the vast majority of media — the billions of photos and videos produced by ordinary people, on their phones, every day — the provenance infrastructure does not exist, and the media is, as before, unauthenticated. The C2PA is, in this sense, a partial solution — it can authenticate some media, but it cannot authenticate all media, and the gap leaves room for the deepfakes and the liar’s dividend to operate.

The Policy Response

The policy response to synthetic media has been, by most accounts, fragmented and incomplete. In the United States, the federal response has been slow. The Deepfake Report Act of 2019 (S. 2065), passed by the Senate in October 2019, required the Secretary of Homeland Security to publish annual reports on deepfakes, but it did not regulate the technology. The National Defense Authorization Act for Fiscal Year 2020 (signed into law on December 20, 2019) included provisions requiring the Department of Defense to address deepfakes, but these were focused on national security applications rather than on the broader information environment. The first federal criminal law on deepfake non-consensual intimate imagery — the TAKE IT DOWN Act — was not signed into law until May 19, 2025.

At the state level, the response has been more active. California, Texas, and Virginia were the first states to pass deepfake laws, in 2019, and by 2025, forty-eight of the fifty states had some form of deepfake legislation on the books. The state laws vary widely — some focus on non-consensual pornography, some on political deepfakes, some on fraud — and the patchwork of state-level regulation has created a complex and inconsistent legal landscape.

In Europe, the EU AI Act includes provisions on synthetic content. Article 50 of the Act, which applies from approximately August 2026 (two years after the Act’s entry into force), imposes transparency obligations on the providers of AI systems that generate synthetic content — requiring them to mark the content as AI-generated and to provide tools for detecting it. The European approach, which focuses on transparency and disclosure rather than on prohibition, is, by most accounts, a more coherent regulatory framework than the fragmented American approach. But the effectiveness of the transparency requirements — whether they will be sufficient to counter the liar’s dividend and the erosion of trust in evidence — remains to be seen.

The Crisis of Evidence

The crisis of evidence — the collapse of the 150-year trust in the photographic record — is, by most accounts, one of the most consequential cultural impacts of the AI revolution. The ability to produce synthetic media that is indistinguishable from real recordings — and the knowledge that such media exists — has undermined the shared assumption that audio and video can be trusted. The assumption was never entirely justified, but it was, for most of the history of photography, good enough. It is no longer good enough.

The crisis is, in some ways, a return to an earlier era — an era before photography, when evidence was testimonial rather than mechanical. In that era, the credibility of a claim depended on the credibility of the person making it, and on the ability of the audience to assess that credibility. The photograph, by providing a mechanical record that was independent of any individual’s testimony, changed the dynamics of evidence. It allowed claims to be evaluated, not on the basis of who was making them, but on the basis of what the record showed. The crisis of evidence is, in some ways, a return to the testimonial era — a world in which the credibility of a recording depends on the credibility of the source, and in which the audience must, once again, assess the source rather than trusting the record.

The return is not, however, a simple one. The testimonial era was a world of limited information — a world in which most people encountered, in the course of a day, a relatively small number of claims, from a relatively small number of sources. The crisis of evidence is unfolding in a world of unlimited information — a world in which people encounter, in the course of a day, thousands of claims, from thousands of sources, through digital media that they do not control and cannot fully assess. The challenge of evaluating evidence in this environment — of distinguishing the real from the synthetic, the trustworthy from the untrustworthy — is, by any measure, one of the defining challenges of the twenty-first century. The technology that created the crisis — the generative AI that produces the synthetic media — is not going away. The question is whether the technologies of authentication, of provenance, and of media literacy can develop fast enough to preserve, in some form, the shared evidence base on which democracy, journalism, and the rule of law depend. The answer is not yet known. The crisis is here.