Nick Bostrom: Superintelligence & the FHI

---

The Swedish Beginning

Nick Bostrom was born on March 10, 1973, in Helsingborg, Sweden. He completed his undergraduate education at the University of Gothenburg, earning a bachelor’s degree in philosophy, mathematics, mathematical logic, and artificial intelligence in 1994. He moved to the University of Stockholm for a master’s degree in philosophy and physics, completing it in 1996. The combination — philosophy, mathematics, physics, logic, AI — was unusual, and it reflected the breadth of Bostrom’s interests. He was not, by temperament, a specialist. He was interested in the big questions — the nature of mind, the future of humanity, the limits of what could be known — and he was willing to use whatever tools were available to address them.

For his PhD, Bostrom moved to the London School of Economics, where he completed his doctorate in the Department of Philosophy, Logic, and Scientific Method in 2000 (supervised by Colin Howson and Craig Callender; thesis: “Observational Selection Effects and Probability”). His doctoral work was on the philosophy of probability and observation selection effects — the technical question of how to reason when the evidence you have is conditioned by the fact that you, the observer, exist. The work was, in some ways, a preparation for the arguments he would later make about the simulation hypothesis and about the risks of superintelligence.

After his PhD, Bostrom took a position at Oxford. He was, by most accounts, an unusual Oxford philosopher — more interested in the future than in the past, more interested in science than in the history of ideas, more interested in influencing policy than in winning intra-disciplinary debates. He was also, by most accounts, unusually productive. He wrote quickly, he wrote clearly, and he wrote on topics that the broader public found compelling. His early Oxford work — on the anthropic principle (the principle that observations must be compatible with the observer’s existence), on human enhancement, on the ethics of extinction — established him as one of the leading voices in a small but growing community of philosophers working on what would come to be called “existential risk.”

---

The Future of Humanity Institute

In November 2005, Bostrom founded the Future of Humanity Institute (FHI) at Oxford, as part of the Faculty of Philosophy and the Oxford Martin School (then the James Martin 21st Century School). The institute was, from the beginning, an unusual academic institution. Its mission was not to study a particular discipline but to study a particular problem: the long-term future of humanity, and the risks that might prevent humanity from having a long-term future. The institute brought together philosophers, economists, physicists, computer scientists, and policy researchers, all working on different aspects of this problem.

The institute’s research covered several areas. The most prominent was existential risk — the study of threats that could cause the extinction of humanity or the permanent destruction of its potential. Bostrom and his colleagues argued that existential risk was, by any reasonable measure, the most important category of risk, because the stakes were unlimited. A catastrophe that killed a billion people was, of course, a terrible thing. But a catastrophe that ended human civilization entirely — that prevented all future generations from existing — was orders of magnitude worse, because it foreclosed not just the lives of those alive at the time but the lives of all who would ever live.

Within existential risk, AI was a central focus. Bostrom had been thinking about the risks of artificial intelligence since the late 1990s, and he had argued, in a series of papers, that the creation of superintelligence — AI that surpasses humans across all cognitive domains — was the most consequential event that could plausibly occur in the twenty-first century. The argument was not that superintelligent AI was likely to be malicious. It was that superintelligent AI, by definition, would be capable of pursuing its own goals, and that if those goals were not aligned with human flourishing, the result could be catastrophic. The alignment problem — the technical problem of ensuring that an AI system’s goals match human values — was, Bostrom argued, the central technical problem of the field.

---

The Simulation Argument

Before Superintelligence, Bostrom was best known for the simulation argument. The argument, published in April 2003 in The Philosophical Quarterly (53(211): 243–255) under the title “Are You Living in a Computer Simulation?”, was a careful piece of probabilistic reasoning that arrived at a striking conclusion. Bostrom argued that at least one of the following three propositions must be true: (1) humanity will go extinct before reaching a “posthuman” stage capable of running detailed simulations of conscious minds; (2) posthuman civilizations will be uninterested in running such simulations; (3) we are almost certainly living in a computer simulation.

The argument was that if (1) and (2) are both false — if humanity survives to a posthuman stage and posthuman civilizations are interested in running ancestor-simulations — then the number of simulated minds would vastly outnumber the number of “real” minds, and the probability that any given mind (including yours) was real rather than simulated would be vanishingly small. The argument captured the public imagination in a way that academic philosophy rarely does. It was discussed in popular science articles, in science fiction, and in the broader cultural conversation about the implications of computation. The argument also established Bostrom’s reputation as a philosopher who could take unusual ideas and argue for them with technical rigor.

---

Superintelligence

Superintelligence: Paths, Dangers, Strategies was published by Oxford University Press in July 2014 (UK) / September 2014 (US). The book was the culmination of a decade of work at FHI, and it was, by most accounts, the most rigorous and comprehensive treatment of the superintelligence question that had been written. The book was organised into three parts, corresponding to the three words of its subtitle. “Paths” examined the ways in which superintelligence might be achieved — through artificial intelligence, through whole brain emulation (scanning a biological brain at sufficient resolution to simulate it in software), through biological cognition enhancement, or through networks of human and machine intelligence. “Dangers” examined the risks — the alignment problem, instrumental convergence (the tendency of intelligent systems to pursue certain intermediate goals, like resource acquisition and self-preservation, regardless of their final goals), the treacherous turn (the possibility that an AI might behave cooperatively while weak and turn hostile once strong). “Strategies” examined what could be done — the relative merits of different approaches to alignment, the governance challenges, the role of international coordination.

The book was widely read in Silicon Valley. Elon Musk called it “worth reading” and cited it as an influence on his decision to co-found OpenAI. Sam Altman, who co-founded OpenAI on December 10, 2015, was among the Silicon Valley readers who took Bostrom’s book seriously. Dario Amodei, who would co-found Anthropic in May 2021, read it. The book became, in the mid-2010s, the canonical text of the AI safety movement. The arguments were not universally accepted — many AI researchers dismissed them as speculative, and many philosophers found them under-argued — but they could not be ignored.

---

The Influence on OpenAI and Anthropic

The influence of Superintelligence on the founding of OpenAI and Anthropic is, by the accounts of the founders themselves, substantial. OpenAI, founded on December 10, 2015 by Sam Altman, Elon Musk, Ilya Sutskever, Greg Brockman, and others, was explicitly motivated by the safety concerns that Bostrom’s book had articulated. The founders were worried that the development of superintelligent AI, if left entirely to profit-driven corporations, might proceed without adequate attention to safety. OpenAI was founded as a non-profit — initially, at least — with the stated mission of building artificial general intelligence for the benefit of humanity.

Anthropic, founded in May 2021 by Dario and Daniela Amodei and six other co-founders (Jared Kaplan, Jack Clark, Chris Olah, Ben Mann, Sam McCandlish, and Tom Brown) who had left OpenAI, was even more directly influenced by the AI safety tradition. The Amodeis had been part of the OpenAI team, and their decision to leave and start a new company was driven, in part, by disagreements about how seriously to take safety. Anthropic was founded as a “safety-focused” AI lab, with the stated mission of building reliable, interpretable, and steerable AI systems. The company’s culture, which emphasised careful thinking about alignment and which was influenced by the effective altruism movement (the movement that applies evidence and reason to doing good), was shaped by the same intellectual tradition that Bostrom had helped build.

---

The 2023 Controversy

On January 9, 2023, Bostrom posted a statement on his personal website titled “Apology for an Old Email.” In the statement, he acknowledged that in an email sent to a philosophy listserv in approximately 1996, he had used a racial slur and had made claims about race and IQ that he now recognised as offensive and misguided. He apologised for the language and the claims, calling the slur “repulsive” and acknowledging that the email reflected views he no longer held.

The apology did not end the controversy. The original email, which was circulating online, contained language and arguments that many found unacceptable, regardless of when they were written. Oxford launched an investigation. Bostrom was reportedly suspended during the investigation. The coverage, in publications including Vice, The Oxford Blue, and the Effective Altruism Forum, was extensive and largely critical.

The controversy intersected, in complicated ways, with broader debates within the effective altruism community. The community, which had been a major source of support for FHI and for AI safety research more broadly, was going through its own crisis following the collapse of the cryptocurrency exchange FTX in November 2022. FTX’s founder, Sam Bankman-Fried, had been one of the largest funders of effective altruism causes, and his arrest and conviction on seven counts of fraud (sentenced to 25 years) had discredited the community’s leadership. Bostrom’s controversy, coming on the heels of the FTX collapse, added to the sense that the effective altruism movement was in trouble.

---

The Closure of FHI

The Future of Humanity Institute was closed by the University of Oxford on April 16, 2024. The decision was, according to most accounts, made by the university administration, citing “administrative headwinds within the Faculty of Philosophy.” The contracts of FHI’s researchers were not renewed in late 2023, and the institute was given a wind-down period before its formal closure. The broader context — the controversy around Bostrom, the FTX collapse, the questions about the effective altruism movement — was widely understood to be a factor.

The closure was, for the AI safety community, a significant loss. FHI had been one of the few academic institutions that took AI safety seriously, and it had trained a generation of researchers who would go on to work at OpenAI, at Anthropic, at DeepMind, and at other labs. The institute’s closure left a gap that would, in time, be filled by other institutions — the Center for Human-Compatible AI at Berkeley (led by Stuart Russell), the Center for AI Safety (a Berkeley-area research institute), the Alignment Research Center (see B37 — Paul Christiano) — but the gap was real, and the loss of FHI was felt across the field.

Bostrom’s own position after the closure was uncertain. His book Deep Utopia: Life and Meaning in a Solved World was published by Ideapress on March 27, 2024, just weeks before FHI closed. He continued to write and to engage with the public conversation, but his role in the AI safety movement was, after 2024, less central than it had been. The movement had grown beyond the institute that had helped found it, and the questions that Bostrom had raised — about superintelligence, about alignment, about existential risk — were now being debated by a much larger and more diverse community, many of whom had never read his work.

---

The Legacy

Bostrom’s legacy is, as of 2026, complicated. He was, by any measure, one of the most influential philosophers of the early twenty-first century. His arguments about superintelligence, about existential risk, about the long-term future of humanity — arguments that had been marginal when he began making them in the late 1990s — had become, by the mid-2020s, central to the way the technology industry thought about AI. The founders of the most important AI companies cited him as an influence. The policy debates about AI regulation were shaped by the framework he had developed. The research field of AI safety, which had been a small and somewhat marginal subfield when FHI was founded, had grown into a substantial industry, with billions of dollars of investment and thousands of researchers.

But the legacy was also, in important ways, contested. Bostrom’s arguments had always had critics, both within philosophy and within AI. Many AI researchers argued that the focus on superintelligence was misplaced — that the real risks of AI were nearer-term and more mundane (bias, displacement, misuse) and that the focus on existential risk distracted from these. The 2023 controversy, and the closure of FHI in 2024, added a personal dimension to these critiques.

The fair assessment, as of 2026, is that Bostrom was right about the importance of the questions, even if he was not always right about the answers. The question of whether superintelligent AI is possible, and if so what risks it poses, is one of the most important questions facing humanity. Bostrom did not invent this question, but he was the first to give it a rigorous, comprehensive treatment, and he was the first to build an institutional infrastructure around it. The infrastructure he built — FHI, the effective altruism community’s focus on AI safety, the network of researchers and policymakers who took the question seriously — outlasted the institution itself. The questions that Bostrom raised will be with us for as long as AI continues to develop. The answers are still being worked out.

---

---

What would change if more people understood the story behind Nick Bostrom? Who benefits from the current state of affairs, and who is left out? The conversation is worth having — with colleagues, with students, with anyone who uses technology without thinking about where it comes from.