FLI Pause Letter, March 2023: 30,000 Signatories

---

The Future of Life Institute

The Future of Life Institute (FLI) was founded in March 2014 by Max Tegmark, an MIT cosmologist; his wife Meia Chita-Tegmark; Viktoriya Krakovna, then a Harvard PhD student; and Jaan Tallinn, the Estonian co-founder of Skype who had become one of the most significant private funders of AI safety research. The institute was based in Cambridge, Massachusetts, and its stated mission was to “steer transformative technology toward benefitting life and away from extreme large-scale risks.”

FLI was, from its founding, an organisation that operated at the intersection of academic research, policy advocacy, and public communication. Tegmark, who had built his career on the physics of the early universe, had become convinced — partly through conversations with Tallinn and partly through his own reading in the Bostrom and Yudkowsky AI safety tradition (see B35 — Nick Bostrom and B36 — Eliezer Yudkowsky) — that the development of artificial intelligence was the most consequential technological trajectory of the twenty-first century, and that the risks it posed deserved serious academic and policy attention. FLI was his vehicle for making that case.

The institute’s early work focused on convening. In January 2015, FLI organised the Puerto Rico AI Conference — a gathering of AI researchers, ethicists, and policymakers that produced an open letter calling for research on the societal impacts of AI. The 2015 letter, signed by Stephen Hawking, Elon Musk, Steve Wozniak, and hundreds of AI researchers, was the predecessor of the 2023 pause letter in both form and substance. It established FLI as an organisation that could mobilise the AI research community around safety concerns, and it set the pattern that the 2023 letter would follow: an open letter, signed by prominent figures, published on FLI’s website, and distributed widely through the press.

Between 2015 and 2023, FLI expanded its activities. It ran grant programmes that distributed millions of dollars to AI safety researchers. It organised conferences on lethal autonomous weapons, on nuclear risk, and on the long-term future of humanity. It published a set of AI safety research priorities that became a reference document for the field. And it maintained a steady stream of public communication — op-eds, podcast appearances, and media briefings — that kept AI risk in the public conversation. By 2023, FLI was one of the most visible AI safety organisations in the world, and Max Tegmark was one of the field’s most quoted public intellectuals. The pause letter was, in many ways, the culmination of eight years of work.

---

The Letter

The letter raised specific concerns: that AI systems could generate propaganda and disinformation at scale, that they could automate jobs faster than new jobs could be created, and that they could eventually pose a threat to human civilisation. The letter invoked the language of existential risk (the threat of extinction) alongside more immediate concerns about economic disruption and democratic integrity.

The letter’s central demand was specific: a six-month pause on the training of AI systems more powerful than GPT-4. During the proposed pause, the letter called for the development of shared safety protocols, advanced and audited by independent outside experts. It called for AI labs to agree that no systems beyond a certain capability threshold should be trained until the protocols were in place. And it called for governments to step in and institute a moratorium if the labs did not voluntarily pause.

The letter also made a broader argument. It compared the AI race to the nuclear arms race of the mid-twentieth century, and it invoked the precedent of the Asilomar Conference on Recombinant DNA — the 1975 gathering at which biologists voluntarily paused genetic engineering research until they could agree on safety guidelines. The Asilomar precedent was deliberate. The letter was not arguing that AI research should stop permanently. It was arguing that the field had reached a moment — the moment when systems became capable enough to be dangerous but not yet capable enough to be understood — where a voluntary pause was both appropriate and historically precedented.

---

The Drafting

The drafting of the letter was, by most accounts, a collaborative process that unfolded over several weeks in early 2023. The core group included Max Tegmark, FLI’s president Anthony Aguirre (a UC Santa Cruz cosmologist), FLI’s policy director Jared Brown, and several external advisors who consulted on the wording. The letter went through multiple drafts, with debates about tone, scope, and specificity.

The decision to call for a six-month pause, rather than an indefinite pause or a more modest set of safety measures, was a compromise. An indefinite pause — the position advocated by Eliezer Yudkowsky (see B36) in his subsequent Time op-ed — was seen as too extreme to attract broad support. A set of safety recommendations, without a specific pause demand, was seen as too weak to be newsworthy. The six-month pause was the middle ground: long enough to be meaningful, short enough to be plausible, and specific enough to be a clear demand.

The decision to focus the letter on GPT-4, rather than on AI systems in general, was also deliberate. GPT-4 had been released on March 14, 2023 — eight days before the letter was published. Its release was, by most accounts, a significant step up in capability from GPT-3.5, which had powered ChatGPT. By anchoring the pause demand to GPT-4, the letter gave itself a concrete reference point — a specific system, with a specific release date, that everyone could identify. The risk, of course, was that “more powerful than GPT-4” was itself undefined, and the letter’s critics would seize on this vagueness.

---

The Signatories

The signatories were a mix of AI researchers, technologists, public intellectuals, and concerned citizens. The most prominent signatories were Elon Musk — who had co-founded OpenAI in 2015 and who had a complex relationship with the company — and Steve Wozniak, the co-founder of Apple. Other notable signatories included Yoshua Bengio, one of the three “godfathers of deep learning”; Stuart Russell, the author of the leading AI textbook; and Yuval Noah Harari, the historian and author of Sapiens.

The signatories also included many working AI researchers and engineers, though some noted that the letter had attracted signatures from people who were not AI experts, which weakened its credibility. The Future of Life Institute did not verify all signatures, and some fake names appeared on the list before being removed. The open signatory process — anyone could add their name — was a deliberate choice. It maximised the number of signatures and the breadth of support, but it also meant that the signatory list could not be used as a precise measure of expert opinion.

The absence of the major AI lab leaders was significant. The 2015 FLI letter had been signed by many AI researchers who worked at the major labs. The 2023 letter, by contrast, was signed almost entirely by people outside the labs — academics, retired researchers, public intellectuals, and concerned citizens. The people building the most advanced AI systems did not sign. This was, in some ways, the letter’s central weakness: it spoke for the concerned but not for the builders.

---

The Definitional Problem

The letter’s most persistent criticism was its vagueness about what “more powerful than GPT-4” actually meant. GPT-4 was a specific system, released on a specific date, with specific capabilities that had been documented in OpenAI’s technical report. But “more powerful than GPT-4” was not a precise category. Did it mean systems with more parameters? Systems trained on more data? Systems that scored higher on specific benchmarks? Systems that could perform specific tasks that GPT-4 could not?

The problem was not merely academic. If the pause had been adopted — if the labs had agreed to stop training systems more powerful than GPT-4 — they would have needed a way to determine, before training a new system, whether it would be more powerful than GPT-4. This would have required a capability evaluation framework — a set of tests that could be run on a model before it was fully trained, to predict whether it would exceed the GPT-4 threshold. No such framework existed in 2023. The field of AI evaluation was in its infancy, and the tools for measuring model capability were rudimentary.

The letter’s supporters argued that this was a reason to pause — that the absence of a capability framework was itself evidence that the field was moving faster than its ability to understand what it was building. The letter’s critics argued that the absence of a framework made the pause demand meaningless — that without a way to define the threshold, there was no way to enforce the pause. Both arguments had merit. The tension between them — between the need for caution and the difficulty of implementing caution — was one of the defining tensions of the AI safety debate, and the pause letter brought it into sharp relief.

---

The Response

The letter was widely covered in the press, and it generated a significant debate. Supporters argued that the letter was a necessary intervention — a call for caution in a field that was moving too fast. Critics argued that the letter was impractical, that a pause would be impossible to enforce, and that it would advantage companies that ignored it over companies that complied.

Sam Altman, the CEO of OpenAI, responded publicly, saying that OpenAI would not pause. Other AI labs also indicated that they would not comply. The competitive dynamics of the AI race made a voluntary pause unrealistic — no company was willing to stop while its competitors continued. The letter was also criticised for its vagueness. The demand for a “pause” on training systems “more powerful than GPT-4” did not specify how “more powerful” would be defined, how the pause would be enforced, or what would happen after the six months. Critics argued that the letter was more of a symbolic gesture than a practical proposal.

Some critics went further. Yann LeCun argued that the letter was actively harmful — that it distracted from the real risks of AI (bias, misinformation, economic disruption) by focusing on speculative existential risks, and that it served the commercial interests of the large AI labs by suggesting that only they could be trusted to build AI safely. Steven Pinker, the Harvard cognitive scientist, called the letter’s existential risk framing “moral panic” and argued that the historical record of technology suggested that AI would be, on balance, beneficial. Timnit Gebru, the AI ethics researcher, argued that the letter’s focus on long-term existential risk diverted attention from the harms that AI was already causing — to marginalised communities, to workers, to democratic institutions.

The range of criticism — from LeCun’s commercial-interest argument to Pinker’s techno-optimism to Gebru’s equity-based critique — illustrated the fragmentation of the AI discourse. The pause letter had brought the discourse to a new level of public visibility, but it had not produced a consensus. If anything, it had made the disagreements clearer.

---

What It Meant

The FLI Pause Letter was, by its own stated criteria, a failure. No AI lab paused. No pause was enforced. The pace of AI development did not slow. The letter did not achieve its stated goal.

But the letter was, by other criteria, a success. It brought the AI safety debate to the front page of every major newspaper. It generated a public conversation about the risks of AI that had not existed before. And it set the stage for the subsequent wave of AI safety interventions — the CAIS Statement in May (see B38), the Altman Senate testimony in May (see B89), the White House voluntary commitments in July (see B90), the Biden executive order in October, and the Bletchley Declaration in November (see B40).

The letter also revealed the central dynamic of the AI safety debate: the tension between those who want to slow down and those who want to speed up. The pause letter represented the “slow down” position. The AI labs represented the “speed up” position. This tension was not resolved by the pause letter. It was not resolved by the subsequent safety interventions. And it has not been resolved since. It is the fundamental tension of the AI era, and it will continue to shape the development of the technology — and the governance of the technology — for years to come.

The letter’s longer-term legacy is more ambiguous. By 2025, the AI safety movement had split. The “pause” position had been marginalised — no major government had adopted a pause, and the major labs had continued to train and release increasingly capable systems. The alignment research programme — the technical work of making AI systems safe, championed by B37 — Paul Christiano — had become the dominant paradigm. The pause letter’s contribution was not that it achieved its stated goal. It was that it made the AI safety debate visible — to the public, to the press, to policymakers — in a way that no previous intervention had. The debate that the letter started has not ended. But the letter itself, as a policy proposal, is now generally understood to have been a gesture — a powerful gesture, a historically significant gesture, but a gesture nonetheless.

---

---

---

The question this piece raises — about the fli pause letter — is not going away. Neither is the need to answer it honestly.