I got roundly ticked off the other day for claiming that the climate crisis was “in a league of its own” when it comes to contemplating causes for the possible extinction of the human race – or, at least, “human civilization”, as we mistakenly refer to the current world order. (‘Existential’ means, in this context, ‘pertaining to existence or the end of existence’).
My reprimand went something like this: “You’re so obsessed with the climate crisis that you’re ignoring equally important threats. Like all-in nuclear warfare. Or AI”.
Well, in my defence, I plead not guilty on ignoring the threat of nuclear war. By contrast, I plead 100%, reprehensibly guilty on the AI charge.
Let me go one by one. Nuclear first.
I remember to this day the talk we were given at school in 1963 about a possible nuclear attack on the UK. This was at the height of the Cold War and shortly after the Cuba missile crisis. It scared the shit out of me.
A few years after that, I joined the Campaign for Nuclear Disarmament and later became obsessed with what are called ‘broken arrows’ – when things go badly wrong in terms of ‘lost’ nuclear warheads, accidents in nuclear facilities, near misses, communication breakdowns and so on. It’s acknowledged that there have been at least 32 of these major incidents — and these are just the ones that we know about!
Every nuclear-armed country will have its own ‘broken arrows’ portfolio. The Nuclear Information Service here in the UK tracks as many of these incidents as it can; back in 2017, its ‘Playing with Fire’ report detailed 127 accidents, near misses and “dangerous occurrences” on UK soil and in its coastal waters since the 1960s. And we know that thousands of cyber-attacks are launched against the Ministry of Defence and its contractors every day. Hair-raising really doesn’t cut it!
So, I guess I’ve been living in a state of measured, back-of-mind nuclear dread for a long time — with the front of my mind increasingly taken up with dread at the thought of the climate apocalypse bearing down on us! But however bad that gets, it would be nothing in comparison to an all-out nuclear war.
So, I’m finding it really painful to see how the notion of Mutually Assured Destruction (the crazy idea that lies at the heart of nuclear deterrence theory) is being gradually ‘re-normalised’. Ideas such as ‘tactical nuclear weapons’ or ‘limited nuclear strikes’ (particularly in terms of the war in Ukraine) pop up all over the place, even as the all-important 1970 Treaty on the Non-Proliferation of Nuclear Weapons (NPT) lurks unregarded in the margins of this new nuclear discourse.
The fact that Article Six (of the NPT) enjoins all 191 signatories to the Treaty ‘to pursue nuclear disarmament in good faith’ is studiously ignored. In fact, exactly the opposite is going on. Today’s nine nuclear-armed nations already lay claim to tens of thousands of ‘Hiroshima bomb equivalents’, and yet this is increasingly seen as insufficient.
China is increasing its arsenal of warheads by around 100 every year. Russia is known to be expanding its capabilities in many different areas. President Trump is already committed to prioritising nuclear weapons programmes over other security programmes, based on $25 billion of additional investment in nuclear weapons every year, with an ambition of reaching $2 trillion on total arms expenditure over 30 years — including massive new expenditure on his so-called ‘Golden Dome’ defensive shield — which essentially undermines the rationale behind any nuclear deterrence theory.
Which brings me to AI.
I enter this territory with massive trepidation. Until recently, when asked about it, I burbled on about the massive energy and water footprint of AI, relying on a basic instinct to cover off the rest of it: that anything emerging from and under the total control of the techno-fascists in California just had to be wrong. Simple as that. Follow the money. Track the track records of these evil mind-fuckers.
Such an approach obviously only takes one so far. Which means I’ve recently been doing some rather more serious work on AI, to which I will no doubt return at some point in the future.
However, for the purposes of this little exercise, I’ve come to the conclusion (from an existential risk perspective) that AI is right up there with both the climate crisis and the threat of nuclear warfare.
Ever since I came across AI’s so-called ‘evolution of capability and complexity’, I’ve been sunk deep in the whole reversible/irreversible dynamic: at what point does a rapidly evolving change process cease to be reversible as it ‘tips over’ into a sequence of irreversible effects?
Climate wonks live with that dynamic every waking moment of their working lives: feedback loops; sensitivity calculations; tipping points – 40 years of brilliant climate science has given us an eloquent lexicon of how to avoid the apocalypse. We approach it via ‘average temperature increase’, average concentrations in the atmosphere, potential tipping points for critical ecosystems etc, etc. Tragically, this seems to have had zero impact on our fossil-fuel-addicted political leaders. Suffice to say, it’s not looking good.
So, is there an analogy for AI? Sort of. As AIs evolve from bog-standard LLMs (Large Language Models) to ANI (Artificial Narrow Intelligence) to AGI (Artificial General Intelligence) to ASI (Artificial Superintelligence) to CASI (Conscious Artificial Superintelligence), one is bound to ask: at what point is that forward momentum still reversible?
This was the point at which I discovered the whole wondrous literature about the so-called ‘X risk’ – which, happily, has nothing to do with the world’s first totalitarian trillionaire. I quote from a fascinating article by David Rollo:
“Consider that Geoffrey Hinton (the so-called ‘godfather of AI’) gives a 10 to 50% chance that AI would result in extinction – the ‘ X risk. Yoshua Bengio, the most cited AI researcher and most cited living scientist, gives an X risk of 20% .
Eliezer Yudkowsky (co-author of the bestseller ‘If Anyone Builds It, Everyone Dies’, a title relating directly to AGI), and Roman Yampolskiy give an X risk of over 95%, while Dario Amodei, CEO of Anthropic, gives an X risk of 10 to 25%”.
Given the single-minded, profit-obsessed drive of the so-called ‘frontier AI companies’ (OpenAI, Anthropic, Google DeepMind and XAI), there would appear to be no political regulatory constraints on them cracking the AGI challenge. Does that turn out to be the not-so-far-off point of irreversibility? On our unstoppable way to the equivalent of all-out nuclear conflagration?
Which makes me think about how things might look in this existential league table – keeping nuclear conflagration in the mix, as it truly deserves to be, even before we start thinking about that deranged, psychopathic narcissist in the White House with his finger on the button.
So, my question for all readers who’ve stuck with my strange apocalyptic speculation so far: where would you locate the point of irreversibility for each of these three domains of existential risk?
And how does that make you feel?
Jonathon Porritt 14 July 2026
