The AI Election: When Synthetic Media Met Democracy

“Washington, D.C. January 21, 2024. New Hampshire residents begin receiving robocalls with an audio recording that sounds exactly like President Joe Biden. The voice — convincingly Biden’s, complete with his characteristic speech patterns and verbal tics — tells New Hampshire Democrats not to vote in the state’s presidential primary. ‘It’s important that you save your vote for the November election,’ the voice says. ‘Voting this Tuesday only enables the Republicans in their quest to elect Donald Trump again.’ It is fake. The voice is AI-generated. The message is fabricated.”

Washington, D.C. January 21, 2024. New Hampshire residents begin receiving robocalls with an audio recording that sounds exactly like President Joe Biden. The voice — convincingly Biden’s, complete with his characteristic speech patterns and verbal tics — tells New Hampshire Democrats not to vote in the state’s presidential primary.

“It’s important that you save your vote for the November election,” the voice says. “Voting this Tuesday only enables the Republicans in their quest to elect Donald Trump again.”

It is fake. The voice is AI-generated. The message is fabricated. And it is indistinguishable, to most listeners, from an actual recording of the President of the United States.

The New Hampshire robocall is the most visible single incident in what will become the defining political story of 2024: the deployment of AI-generated synthetic media — voice clones, deepfake videos, AI-generated text — in political campaigns, in foreign influence operations, and by individual bad actors attempting to manipulate voters and undermine democratic processes.

This is the story of the first AI election — the election cycle in which the technology that had been developing for decades, and that had entered public consciousness with ChatGPT in 2022, collided with the most fundamental institution of democratic society.

---

Before 2024: The Accumulated Warning Signs

The 2024 election cycle was not the first time AI-generated content had appeared in a political context. The warning signs had been accumulating for several years.

In 2018 and 2019, the deepfake phenomenon — AI-generated videos that placed real people’s faces onto other people’s bodies — had attracted significant attention, primarily in the context of non-consensual intimate imagery. The specific threat to political discourse was identified early: if the same technology that could place a celebrity’s face on a pornographic video could place a politician’s face in a fabricated political speech, the implications for democratic discourse were serious.

The specific risk was the “liar’s dividend” — the risk that even genuine videos of politicians could be dismissed as deepfakes, undermining the evidentiary value of all video in political discourse. In an environment where AI-generated video was common and convincing, any video that a politician found inconvenient could be dismissed as synthetic. The presence of deepfake technology degraded the reliability of all video evidence, not just the specific fakes.

In 2019, Congress held hearings on deepfakes. Several states passed laws specifically criminalising deepfakes used in political advertising. The Department of Defense’s DARPA program funded research into deepfake detection technology. The awareness of the threat was real and the legislative response was beginning.

In 2023, following ChatGPT’s launch and the proliferation of accessible text-to-image and text-to-audio generation tools, the scale of the concern increased. Several incidents during the 2023 off-year elections involved AI-generated content. A deepfake of a Slovakian political figure appearing to discuss election fraud circulated widely in the days before the Slovakian parliamentary election. The incidents demonstrated that the technology had advanced to the point where sophisticated deepfake creation was accessible to individuals with modest technical sophistication and access to consumer AI tools.

---

The 2024 Landscape: A Perfect Storm

The 2024 election cycle was a perfect storm for AI-generated political manipulation, for several reasons that combined to create a uniquely challenging environment.

The technology had crossed a threshold. Voice cloning technology had become accessible and convincing. Text-to-image and text-to-video generation was more capable than in previous election cycles. The tools required to generate convincing synthetic media were available through consumer applications at minimal cost.

The political environment was intensely polarised. In the United States, the 2024 cycle was the third consecutive presidential election to involve Donald Trump, in a political environment of extraordinary polarisation. Voters were primed to believe the worst about political opponents, and synthetic media that confirmed existing negative beliefs about political figures was likely to be shared widely without fact-checking.

Social media distribution amplified speed over accuracy. The social media platforms through which most voters consumed political information were optimised for engagement rather than accuracy — content that generated strong emotional reactions was distributed more widely, regardless of its truthfulness.

Defensive infrastructure was inadequate. The technology for detecting AI-generated content was significantly behind the technology for generating it. Detection systems that worked for the previous generation of deepfake technology were being defeated by the current generation.

The regulatory framework was incomplete. The laws that governed deepfakes in political advertising were a patchwork — some states had enacted specific deepfake disclosure or prohibition laws, others had not. Federal law had not been updated to address AI-generated political content.

---

The New Hampshire Robocall: A Case Study

The New Hampshire robocall became the most analysed single AI manipulation incident of the election cycle, and examining it carefully reveals both the capabilities and the limitations of AI political manipulation.

The audio was generated using an AI voice cloning service — technology commercially available through several providers for a subscription fee of a few dollars per month. The sample audio used to clone Biden’s voice was drawn from publicly available recordings of the President speaking. The manipulation required no sophisticated technical expertise.

The targeting was specific: New Hampshire Democratic primary voters, reached at the specific moment when their participation in the primary could affect the results. The message was designed to confuse and discourage — to tell Democratic voters that participating in the primary was somehow counterproductive to their actual political goals.

The robocall was traced, within weeks of its circulation, to a political consultant with connections to a candidate running against Biden in the primary. The consultant was eventually charged with election interference and impersonating a federal official.

The speed of the trace was important. The incident demonstrated that AI-generated political manipulation was not untraceable — that the digital forensics infrastructure could produce results within weeks. But weeks was too slow to prevent the content from reaching its intended audience, and the harm occurred before the trace was complete.

The New Hampshire case illustrated the asymmetry at the heart of the AI election manipulation problem: generating synthetic media is fast, cheap, and requires minimal expertise; detecting it, tracing it, and legally addressing it is slow, expensive, and requires significant institutional resources.

---

The Anatomy of AI Political Misinformation

The broader landscape of AI-generated political content in the 2024 cycle was extensive and varied, involving different types of content and different actors.

State-sponsored influence operations. Foreign governments — primarily Russia, China, and Iran — used AI tools to enhance the scale and sophistication of their influence operations. AI added several specific capabilities: scale (generating vast quantities of content at a pace humans could not match), sophistication (grammatically correct, culturally fluent text), and personalisation (content targeted to specific audiences and news moments).

Domestic political operatives. The New Hampshire robocall demonstrated that domestic political actors were also deploying AI tools. Several incidents involved AI-generated images of political candidates in unflattering contexts, AI-generated audio clips of candidates making statements they had not made, and AI-generated text designed to mislead specific voter communities.

Individual bad actors. The accessibility of AI generation tools meant that individual bad actors with no connection to organised political operations could generate and distribute synthetic political content. The barriers to creating a realistic-sounding audio clip or an image of a political candidate in a compromising situation were, by 2024, accessible to anyone with a smartphone and a few minutes of time.

---

The Global Dimension: AI Elections Around the World

The 2024 election cycle was not limited to the United States. More than sixty countries held significant elections in 2024, and AI-generated content appeared in many of them.

India. The Indian general election — the largest democratic exercise in human history, with nearly a billion eligible voters — was the most significant single AI election event outside the United States. AI-generated content appeared extensively: deepfake videos of political figures, voice clones used in political appeals, AI-generated images in political advertising. One notable development was the use of AI to generate content in regional languages, enabling political operations to target diverse linguistic communities at unprecedented scale.

Bangladesh. The Bangladeshi election involved AI-generated deepfakes of opposition politicians that circulated on social media, designed to embarrass opposition figures and discourage their supporters.

Pakistan. Imprisoned former prime minister Imran Khan’s party used AI-generated audio and video — including AI-generated speeches that sounded like Khan — to campaign on his behalf, raising questions about the boundary between legitimate political communication and deceptive synthetic media.

Taiwan. The Taiwanese presidential election, occurring in the context of cross-strait tensions, involved sophisticated AI-generated disinformation attributed to Chinese influence operations, including AI-generated audio of political figures and AI-enhanced social media operations.

The global pattern demonstrated that AI election manipulation was a challenge for democratic processes globally, wherever accessible AI generation tools existed and wherever elections were conducted.

---

The Defensive Landscape: What Worked and What Didn’t

The response to AI election manipulation in 2024 involved multiple actors deploying a range of defensive measures with varying degrees of effectiveness.

Content detection. Technology platforms deployed AI-based tools for detecting synthetic media and removing it before or after viral distribution. The effectiveness was limited by the fundamental asymmetry: detection technology consistently lagged behind generation technology. By the time detection models were trained on one generation of synthetic media, the next generation of generation technology had produced content that defeated the detection.

Content authentication. The C2PA standard, which embeds cryptographic provenance information in media files, was adopted by several major camera and software manufacturers. But adoption was far from universal, and most synthetic content that circulated in the 2024 cycle did not carry authentication information.

Platform policies. The major social media platforms updated their policies regarding AI-generated political content. Meta required disclosure when political ads used AI-generated images or audio. Google imposed restrictions on AI-generated political advertising on YouTube. X’s policies on synthetic media were less consistent and enforcement was less reliable.

Legal frameworks. Several US states enacted or expanded laws governing AI-generated political content, primarily through disclosure requirements. The federal government proposed but did not pass specific federal legislation. The constitutional constraints on political speech created specific challenges for more restrictive federal approaches.

AI company voluntary commitments. The major AI companies made voluntary commitments not to support AI-generated election manipulation, including declining to generate content impersonating political candidates and adding watermarking to AI-generated content. The effectiveness of voluntary commitments was limited by their inability to cover the rapidly proliferating ecosystem of open-source models and consumer tools.

---

The Voter Response: Did It Work?

The fundamental question about AI election manipulation is whether it actually influenced voters — whether the synthetic media changed election outcomes.

The honest answer is: we do not know with confidence. Measuring the influence of specific pieces of misinformation on voter behaviour is methodologically challenging.

Several observations are possible. The AI-generated content that circulated in 2024 reached significant audiences in some cases but had limited reach in most cases relative to the total volume of political information voters consumed. Research on the influence of misinformation suggests it is most influential when it confirms existing beliefs. The 2024 election outcomes in the United States and most other countries were broadly consistent with polling and other predictors, suggesting that AI manipulation — if it had effects — was not large enough to reverse outcomes.

The more insidious potential effect of AI election manipulation is not on individual election outcomes but on the broader epistemic environment — on voters’ trust in political information, their willingness to engage with the political process, and their ability to distinguish truth from fabrication. These effects are harder to measure than outcome effects but may be more significant in the long run.

---

The Democratic Theory Question: What Does AI Do to Democracy?

The AI election challenge raises a specific question in democratic theory: what does democracy require to function, and is AI-generated manipulation a fundamental threat to those requirements?

Democratic theory traditionally requires several conditions for legitimacy. Citizens need access to accurate information about candidates and policies. Citizens need to be able to engage in genuine deliberation. Elections need to produce outcomes that reflect the genuine preferences of voters, not preferences manufactured by manipulation.

AI-generated political manipulation potentially threatens each of these conditions. A proliferation of convincing synthetic media degrades the information environment. Undermining agreement on basic facts — what a candidate has said, what events have occurred — degrades the shared epistemic foundation that deliberation requires. If AI-generated manipulation can convincingly misrepresent what candidates have said and done, the preferences that elections express may be based on false beliefs.

These concerns are real and serious. But they need to be situated in context. Democratic discourse has never been free of manipulation, distortion, and the strategic presentation of misleading information. The question is whether AI-generated manipulation represents a departure large enough in degree to become a difference in kind.

The case that AI manipulation is different in kind rests on two arguments: the scale argument (AI enables manipulation at a scale that manual human efforts cannot match) and the authenticity argument (AI generates content that can pass as genuine speech by specific individuals in ways that previous manipulation could not). Both arguments have force, and together they suggest a specific challenge to democratic processes that deserves serious governance attention.

---

The Long-Term Challenge: Democracy in the AI Era

The 2024 election cycle was not the end of the AI election challenge — it was the beginning. The capabilities deployed in 2024 will be more sophisticated in 2026 and more sophisticated still in 2028.

Building AI-era epistemic infrastructure — authentication systems, provenance tracking, trusted information intermediaries — is a challenge that goes beyond technical detection. It requires institutional changes to how information is produced, distributed, and evaluated.

Citizens need to be equipped to navigate an information environment in which AI-generated synthetic media is common. The specific skills required — questioning the provenance of media, seeking verification before sharing, recognising the specific tells of synthetic media — are not part of the traditional civic education curriculum.

The regulatory frameworks that govern AI-generated political content need to develop alongside the capabilities they are governing. The patchwork of voluntary commitments and state-level legislation that characterised the 2024 response is inadequate for the scale and sophistication of the challenge.

The social media platforms need to be held accountable for their role in distributing AI-generated political manipulation in ways that current regulatory frameworks do not provide. The platforms’ commercial incentives, which partially misalign with healthy democratic discourse, need to be addressed through governance mechanisms that create genuine accountability.

The first AI election revealed the challenge. Meeting it will be one of the defining political tasks of the next decade.

---

Further Reading

---

---