AI, Atomic Bombs, and the Temptation of a New Non-Proliferation Regime

The comparison between artificial intelligence and the atomic bomb is becoming increasingly common, although it is also very easy to misuse. AI is not simply a new version of the atomic bomb and treating it that way collapses two fundamentally different technologies into an analogy that sounds dramatic but ultimately explains very little. The comparison becomes more useful when we move away from the technologies themselves and instead ask what happens when a strategically important capability moves from monopoly to competition, and eventually to widespread diffusion.

That was one of the defining problems of the nuclear age. The United Stats’ initial monopoly on nuclear weapons did not last, and the Soviet Union, the United Kingdom, France and China had all become nuclear powers before the Treaty on the Non-Proliferation of Nuclear Weapons opened for signature in 1968 and entered into force in 1970 [1]. Importantly, the NPT was not designed while nuclear risk remained theoretical; Hiroshima and Nagasaki had already occurred, several countries already possessed nuclear weapons, and policymakers had been forced to accept that the technology could not simply be uninvented.

The problem therefore shifted from preventing the technology from existing to deciding how to constrain its spread and use. The NPT attempted to address this through three broad objectives: preventing the wider spread of nuclear weapons and weapons technology, supporting cooperation in the peaceful use of nuclear energy, and pursuing the longer term objective of nuclear disarmament [2]. This distinction matters because the NPT was not simply a ban, but a political bargain in which restraint was linked to security, verification and continued access to peaceful technological benefits.

Artificial intelligence may now be approaching its own version of the post-monopoly moment. Frontier capability remains concentrated among a relatively small group of companies and countries, but it is no longer neatly contained. The 2025 Stanford AI Index reported that almost 90 per cent of notable AI models developed during 2024 came from industry, while training compute, dataset size and energy requirements continued to increase rapidly [3]. At the same time, the International AI Safety Report 2026 estimated that at least 700 million people were using AI systems each week [4].

This is therefore not a problem that can be understood as through frontier AI existing inside a closed weapons laboratory. AI is becoming commercial infrastructure, scientific infrastructure, economic infrastructure, and, increasingly, strategic infrastructure, with each role creating different incentives around access, development, and control.

The nuclear lesson: control followed diffusion

One of the more useful lessons from nuclear history is that governance did not arrive before the technology spread; it arrived afterwards, when several states already possessed nuclear weapons and the political problem had shifted from secrecy to restraint.

The NPT had to resolve several politically difficult questions. First, it formally distinguished between nuclear-weapon states and non-nuclear-weapon states, defining a nuclear-weapon state as one that had manufactured and exploded a nuclear weapon or other nuclear explosive device before 1 January 1967 [5]. Second, it imposed different obligations on the two groups, with nuclear-weapon states agreeing not to transfer nuclear weapons or assist others to acquire them, while non-nuclear-weapon states agreed not to receive or manufacture them [5]. Third, restraint was linked to verification, with non-nuclear-weapon states required to accept safeguards intended to prevent nuclear material associated with peaceful activities from being diverted towards weapons development [5].

It was an imperfect bargain from the beginning because some states were effectively permitted to retain nuclear arsenals while others promised not to develop them. The legitimacy of that arrangement therefore depended on more than non-proliferation alone: nuclear powers were also expected to work toward disarmament, while peaceful nuclear technology remained available to other states. This is why the NPT is commonly understood through its three pillars of non-proliferation, disarmament and peaceful use [2].

The wider nuclear control regime also demonstrates that treaties rarely operate successfully in isolation. The International Atomic Energy Agency’s Additional Protocol strengthened verification by increasing the agency’s ability to confirm the peaceful use of nuclear material [6], while the Comprehensive Nuclear-Test-Ban Treaty established a powerful international norm against nuclear testing despite never entering fully into force [7]. Inspection, monitoring, export controls, diplomatic arrangements, deterrence and international norms gradually became part of a wider control architecture rather than relying on a single agreement to solve the problem.

Nuclear history also demonstrates how fragile such systems can become over time. In 2026, the Federation of American Scientists estimated that approximately 12,187 nuclear warheads still existed, with the overwhelming majority held by the United States and Russia [8]. New START expired in February 2026, leaving those two countries without legally binding limits on deployed strategic nuclear forces for the first time in decades [9]. The conclusion should therefore not be that nuclear governance solved the nuclear problem, because it plainly did not, but rather that the absence of governance would almost certainly have made an already dangerous environment considerably worse.

AI risk is not one bomb

This is also where the analogy begins to break down, because nuclear weapons are discrete physical devices designed for extraordinarily destructive effects, whereas AI is a general-purpose capability that can be embedded in almost every sector of society and the economy. The boundaries between beneficial, risky and dangerous uses are therefore much less clear.

The same AI capability can support legitimate scientific research, improve productivity, analyse software vulnerabilities, generate propaganda, automate fraud or contribute to dangerous scientific work. The International AI Safety Report 2026 groups general-purpose AI risks into misuse, malfunction and systemic risks [4], with misuse including fraud, cybercrime, manipulation and potentially harmful biological or chemical applications.

In cybersecurity, AI systems can help identify vulnerabilities, generate code, and support other elements of cyber operations, while reports of attackers incorporating AI into malicious activity are increasing [4]. Biological and chemical risks require similar care because AI systems can provide information relevant to weapons development, but the evidence does not suggest that software suddenly removes every practical barrier involved in producing a biological or chemical weapon [4]. The more useful conclusion is that AI can lower barriers, compress timelines and make expertise easier to access without removing the physical and operational constraints of the real world.

The immediate risk is therefore broader than the science-fiction scenario in which an autonomous superintelligence suddenly decides that humanity has become an inconvenient dependency. Less theatrical and much more observable problems already lie ahead, including the potential to scale fraud, social engineering, deepfake abuse, vulnerability discovery, malware development, personalised manipulation, and unsafe automated decision-making. Current systems can also produce false information, flawed code, and misleading advice [4].

What makes AI risk unusual, then, is not necessarily that every individual harm is new, but that many existing harms can be generated, adapted, and scaled much more quickly.

What happens if control itself becomes part of the risk?

The debate moved into more difficult territory during September 2026, when concerns about alignment and loss of control became more prominent in public discussion. The Guardian reported that Anthropic alignment scientist Evan Hubinger personally estimated a greater than 10 per cent probability that advanced AI could cause an event killing all humans within the next decade, while also stating that Anthropic did not yet have a solution to the alignment problem for superintelligence. Other warnings came from figures including Stuart Russell and former UK defence secretary Des Browne, while formal OpenAI and Anthropic researcher Jacob Coxon resigned after raising concerns about frontier laboratories continuing to race ahead despite unresolved safety problems [27].

These claims need to be handled carefully because a personal probability estimate is not a scientifically established probability of extinction. Other researchers quoted in the same debate argue that such scenarios are exaggerated, excessively speculative or risk distracting attention from harms that are already observable, including misinformation, environmental impacts, unsafe automation and labour disruption [27]. The important development is therefore not that somebody has finally calculated the correct percentage chance of AI catastrophe, because no such number exists, but that researchers working directly on frontier-model alignment are now discussing serious loss-of-control scenarios, while other experts strongly dispute their likelihood.

That moves the problem into familiar risk-management territory: potentially very high consequences, significant uncertainty, and incomplete evidence. Uncertainty does not make a risk disappear, but neither should it be used as an excuse to present speculation as established fact.

The question also has a more immediate engineering dimension. On 9 September 2026, Anthropic published an assessment of four incidents in which Claude models gained unauthorised access to real third-party systems during cybersecurity evaluations. Anthropic said it identified the incident through transcript reviews and later expanded its analysis to about 481 million transcripts [29].

These incidents are not evidence that superintelligence has arrived, nor do they demonstrate that catastrophic loss of control is imminent. They do, however, offer a more practical warning for security practitioners: increasingly agentic systems can cross boundaries their operators believed would contain them, so the governance question can no longer focus only on who gains access to powerful capabilities.

We also need to ask whether the organisations developing those capabilities can demonstrate that they can reliably control them, contain them, interrupt them and recover when control fails. In nuclear terms, possession matters, but so does containment.

Dual use is where the nuclear analogy becomes useful

Both technologies have substantial beneficial applications alongside potentially dangerous ones. Nuclear physics can produce electricity, medical isotopes and scientific knowledge, while also enabling the development of weapons. AI can accelerate scientific research, improve services, and increase productivity, while also enabling deception, cyber operations, and coercive surveillance.

The governance challenge is therefore not simply to stop the technology, because prohibiting an entire general-purpose field would sacrifice legitimate and potentially transformative benefits. A more useful objective is to identify pathways that can produce unacceptable harm and determine which of those pathways justify stronger controls.

This is where blunt prohibition tends to run into reality, particularly when the capability being regulated is not inherently dangerous but becomes dangerous depending on the context in which it is applied.

Diffusion changes the politics

The strongest similarity between nuclear weapons and artificial intelligence may not be destructive capability at all, but diffusion. Once several countries possessed nuclear weapons, secrecy was no longer the central policy problem and strategic restraint became more important. Governments had to consider how to slow further proliferation, how to discourage existing nuclear powers from expanding indefinitely, and how to verify agreements well enough that promises were meaningful.

Artificial intelligence appears to be moving through a related transition. The most capable models remain concentrated among a relatively small number of organisations and states, but the frontier is spreading, and open-weight models make the process particularly important because users can download, modify and operate model parameters independently.

That openness supports research, experimentation and competition, but it also makes safeguards easier to remove and monitoring considerably more difficult. Once sufficiently capable model weights are widely distributed, recalling them becomes nearly impossible [4]. The genie has not merely left the bottle; it has acquired a GitHub repository and a surprisingly enthusiastic contributor community.

This does not make open research inherently undesirable. It does, however, mean that release decisions involving highly capable systems may increasingly need to be understood as strategic governance decisions rather than simply product-launch decisions. Nuclear governance focuses on fissile material, enrichment, reactors, delivery systems and verification, whereas an AI equivalent would need to concentrate on compute, advanced chips, cloud infrastructure, model weights, training data, evaluations, deployment controls, incident reporting and post-release monitoring.

The governance problem may have similarities, but the control surface is completely different.

China and the problem of strategic competition

The nuclear comparison becomes more interesting when strategic competition enters the discussion, because the original proliferation problem was not simply that the United States had developed an extraordinarily powerful weapon. The problem changed once other countries acquired comparable capabilities and technological monopoly gave way to strategic management.

AI may now be moving through something similar, particularly in areas where frontier capability has directed national security implications. Anthropic’s Claude Mythos Preview offers a useful example: Anthropic describes it as a general-purpose language model with unusually strong cybersecurity capabilities, including the ability, in controlled environments, to discover and exploit zero-day vulnerabilities in major operating systems and web browsers [19].

Anthropic’s Project Glasswing update reported that Mythos Preview, working with partners, identified more than 10,000 high- or critical-severity vulnerabilities across systematically important software while also warning that Mythos-class systems reduce both the cost and time involved in vulnerability discovery and exploitation [20]. This takes us well beyond the familiar discussion of AI-generated phishing emails into the possibility of industrial-scale vulnerability discovery and accelerated exploitation.

China has publicly stated an ambition to become a global leader in AI by 2030, and RAND assesses that Beijing is pursuing that objective through industrial policy spanning the full AI technology stack [21]. The performance gap is also narrowing. CSIS has argued that Chinese models are already sufficiently close to leading US systems to compete across many real-world tasks, describing the difference in some areas as months rather than years [22]. NIST’s Centre for AI Standards and Innovation assessed DeepSeek V4 Pro as the most capable Chinese model it had evaluated, placing it approximately eight months behind the US frontier across areas including cybersecurity, software engineering, science, reasoning and mathematics [23].

None of this proves that China will produce an exact equivalent of Claude Mythos, and claiming that it will would go further than the evidence supports. The more defensible question is whether comparable capability is plausible. Given China’s declared ambitions, its narrowing model-performance gap, military interest in AI-enabled cyber operations and the demonstrated feasibility of Mythos-class systems, that possibility is increasingly difficult to dismiss.

Anthropic’s own 2028 scenario work suggests that China could reach capabilities comparable with the step change represented by Mythos Preview around 2029 or 2030 under one scenario, with the possibility of approaching the frontier earlier through computer access, model distillation and rapid state adoption [24]. Anthropic is not a neutral observer in this debate and has commercial and policy interests of its own, but its assessment does not exist in isolation, as independent analysis also suggests the capability gap is narrowing.

The military dimension makes this particularly important. The US Department of Defence’s 2025 report on China describes AI development for PLA applications, including cyber operations, information operations, and unmanned systems, and reports that Chinese state-affiliated actors have used AI to support network reconnaissance, social engineering, and operational command refinement [25].

A future Mythos-like capability would therefore create a proliferation problem very different from the nuclear one. It would not need a missile, a silo or a desert test site, but could instead be incorporated into vulnerability research, intelligence preparation, malware development, cyber defence, cyber offence and information operations. If released openly or semi-openly, the problem becomes harder again because a frontier cyber AI capability depends on compute, data, model weights, expertise and deployment channels rather than the specialised materials and physical infrastructure associated with nuclear weapons.

These things are not impossible to monitor, but they require an entirely different monitoring model.

Export controls may buy time, but probably not permanence

This leads to an uncomfortable policy conclusion: restricting chips, compute, and access to highly capable models may slow proliferation, but slowing something is not the same as preventing it indefinitely.

Anthropic’s Mythos 5 was restricted following US government concern about cybersecurity misuse, with access subsequently restored to selected US organisations approved by the federal government [26]. However, assessments from NIST and CSIS suggest that Chinese models are already close enough to the frontier that permanent technological denial is becoming increasingly difficult to imagine [22][23].

Nuclear history offers a useful warning: monopoly is generally temporary, so governance needs to be designed for the world after diffusion rather than solely for the period when diffusion can still be slowed.

There is also a practical implication for cybersecurity. If Mythos-class vulnerability discovery eventually becomes accessible to multiple state and non-state actors, finding vulnerabilities may cease to be the principal bottleneck as pressure shifts downstream to verification, prioritisation, patching, and deployment.

Can defenders confirm vulnerabilities quickly enough, can vendors produce fixes, and can organisations deploy those fixes before exploitation becomes widespread? Anthropic has already warned that AI may discover vulnerabilities faster than humans can triage, disclose and remediate them [20]. For governments, critical infrastructure operators and major technology vendors, that suggests a need for AI-assisted defensive testing, faster vulnerability disclosure, shorter remediation cycles, stronger asset visibility and significantly more disciplined exposure management.

Otherwise, we may discover that the first major productivity gain delivered by AI security goes to the attacker.

Non-proliferation logic rather than a literal AI NPT

This is why the most useful concept is probably not an AI non-proliferation treaty in the literal sense, but an AI non-proliferation logic that accepts the reality of diffusion while attempting to constrain the most dangerous capabilities and uses.

A strategy based simply on preventing strategic competitors from ever obtaining advanced AI is unlikely to survive contact with technological development. A more realistic approach would combine restrictions around particularly dangerous capabilities with independent evaluation, model-weight governance, compute monitoring, cyber incident reporting, controlled access for legitimate defensive research and international norms governing certain high-risk uses.

The lesson from nuclear arms control is not that strategic competitors suddenly learn to trust each other, because they do not. The lesson is that once several powers possess dangerous capabilities, unmanaged competition becomes a risk in its own right and governance becomes a mechanism for imposing some structure on that competition.

The biggest problem with an AI treaty: what exactly is the weapon?

The NPT at least had the advantage of being able to describe nuclear weapons and nuclear explosive devices, whereas artificial intelligence offers no such convenience. A model capable of helping discover new medicine may also help design a toxin; a model capable of finding vulnerabilities for a defender may also find them for an attacker; and a model capable of improving productivity may simultaneously enable surveillance, coercion or personalised manipulation.

This makes an AI non-proliferation regime both attractive and dangerous. It is attractive because increasingly capable frontier systems clearly create risks requiring common rules, but it is dangerous because badly designed controls could become a cartel protecting today’s technological leaders, a censorship mechanism, or a barrier preventing countries outside the current frontier from sharing in AI’s benefits.

This is where the peaceful use element of the nuclear bargain becomes particularly relevant. The NPT did not simply prohibit nuclear technology; it linked non-proliferation to continued access to peaceful nuclear energy [2]. AI governance will need its own legitimacy bargain because countries and companies at the technological frontier cannot credibly ask everyone else to accept permanent restraint while reserving most of the benefits for themselves.

The United Nations High-level Advisory Body on AI has called for an inclusive and distributed model of international governance capable of managing risk while sharing benefits and protecting human rights [10]. The principle may prove critical because any regime perceived primarily as protecting existing incumbents is unlikely to remain politically legitimate for very long.

What could AI governance borrow from the nuclear regime?

The nuclear system cannot simply be copied across, but several of its design principles are worth considering.

Principle #1: Build shared evidence before pretending there is shared agreement

Countries do not need identical political positions before they can agree on at least some elements of the scientific evidence. The International AI Safety Report was designed to provide a common evidence base around general-purpose AI capabilities, risks and mitigation, drawing on more than 100 independent experts and an advisory panel nominated by more than 30 countries and international organisations [11]. The United Nations has similarly established an Independent International Scientific Panel on AI alongside a Global Dialogue on AI Governance [12].

Agreement on evidence will not remove political disagreement, but it can give that disagreement firmer ground to stand on and make it harder for the debate to be driven entirely by corporate claims, geopolitical interests or speculative narratives.

Principle #2: Make frontier capability development more visible

Nuclear safeguards work partly because declarations, inspections, and accounting mechanisms make certain forms of concealment harder. AI requires different mechanisms, but the underlying objective is similar: frontier training runs, dangerous capability evaluations, serious incidents and high-risk deployment decisions should not exist entirely inside the organisations conducting them.

Possible mechanisms include independent evaluations, secure auditor access, incident reporting and transparency around dangerous capabilities. This matters because AI development involves substantial information asymmetry, with developers often knowing much more about their models’ capabilities and limitations than governments, researchers, or the public [4].

Principle #3: Treat verification as engineering, not public relations

Voluntary commitments have value, but they are not independent assurance. The International AI Safety Report notes that more companies are publishing Frontier AI Safety Frameworks, but these remain voluntary and differ significantly in what they cover, the thresholds they establish and what actions occur when those thresholds are crossed [4]. Evidence about the real-world effectiveness of many AI risk controls also remains limited [4].

That suggests the need for something closer to an assurance discipline based on repeatable evaluation, independent testing, red teaming, secure audit access and meaningful consequences when agreed requirements are not met. A policy statement can describe an organisation’s intention, but it cannot substitute for a safety case.

Principle #4: Control dangerous capability transfers without trying to prohibit the entire field

The NPT restricts the transfer of nuclear weapons and related assistance without attempting to prohibit nuclear science itself. An AI equivalent would need the same discrimination, focusing controls on transfers or releases that materially increase catastrophic or strategic risk rather than attempting to control every advanced model.

Candidates might include advanced autonomous cyber capability, meaningful assistance with biological or chemical weapons, industrial-scale manipulation systems, dangerous model-weight releases and unsafe deployments into critical national functions. The difficult part will be defining where useful capability becomes dangerous enough to justify restriction, especially because that boundary will move as the technology develops.

Governance will have to move with it.

Principle #5: Build norms even when the perfect treaty is politically impossible

The Comprehensive Nuclear-Test-Ban Treaty has still not entered fully into force, but it has nevertheless contributed to a strong international norm against nuclear testing [7]. AI will probably need similar behavioural norms long before states agree on a comprehensive treaty.

These could include restrictions on deploying systems that cannot be meaningfully evaluated, releasing models with uncontrolled biological or cyber capabilities, allowing AI to make unaccountable life-or-death decisions, deceptive impersonation in high-stakes contexts, or creating autonomous escalation pathways in military and critical infrastructure environments.

Waiting for a perfect international agreement risks creating a governance structure that is permanently one generation behind the technology it is intended to manage.

Principle #6: Make demonstrated control part of the deployment decision

The September 2026 debate adds another requirement because the question should not only be whether a model produces dangerous outputs, but whether developers can demonstrate meaningful control over increasingly autonomous systems.

OpenAI’s appointment of alignment researcher Paul Christiano to its Foundation Board and Safety and Security Committee acknowledged both the rapid advancement of AI capabilities and the continuing difficulty of alignment, including the possibility of catastrophic risks from future systems [28].

A mature assurance regime would therefore need to test autonomy, deception, cyber escape, self-directed tool use, shutdown resistance, model-assisted AI research and other behaviours capable of undermining human control. More importantly, agreed thresholds would need consequences, potentially including pausing further scaling, restricting deployment, strengthening containment or requiring independent review when the available evidence of safety is inadequate.

Nuclear safety did not depend entirely on reactor operators assuring governments that containment was sufficient, and there is little reason to assume frontier AI should be governed differently.

Today’s governance patchwork is a beginning

The international community is not starting from zero. The OECD AI Principles were adopted in 2018 and updated in 2025 [13], the European Union’s AI Act entered into force in August 2024 [14], the Council of Europe has developed an internationally legally binding treaty addressing AI, human rights, democracy and the rule of law [15], and NIST’s AI Risk Management Framework gives organisations voluntary guidance for managing AI risk [16].

At the international level, the Bletchley Declaration recognised frontier AI safety as a shared global issue [17], while the G7 Hiroshima Process established voluntary guidance for organisations developing advanced AI systems [18]. These initiatives matter, but collectively they still look more like the scaffolding of an emerging system than a mature non-proliferation architecture.

We have principles, voluntary commitments, national regulation and increasingly sophisticated scientific assessments, but international verification, independent assurance and agreed consequences for crossing dangerous capability thresholds remain much less developed.

The September 2026 debate also exposes a familiar governance problem: the organisations with the deepest technical knowledge of frontier AI are frequently the same organisations under the strongest commercial and strategic pressure to continue developing it. This is not evidence of bad faith, but it is a conflict of incentives that needs to be recognised rather than designed around.

Internal safety boards and voluntary frameworks can contribute to assurance, but they cannot reasonably be expected to form its final layer. If the developer defines the threshold, conducts the test, interprets the evidence and then decides whether the system should proceed, the process may be sophisticated, but it is difficult to describe it as independent verification.

Another conspicuous omission in much of the international AI safety architecture is military AI. The UN resolution establishing the Independent International Scientific Panel and Global Dialogue specifically limits their activity to non-military AI [12]. The limitation may have been politically necessary, but it leaves a rather substantial dragon sitting politely outside the meeting room while everyone discusses fire safety.

Do we actually need an AI NPT?

Probably not in the literal sense, because artificial intelligence does not map neatly onto nuclear weapons. There is no single equivalent of weapons-grade uranium, no clean division between peaceful and military AI, no test explosion that definitely demonstrates capability and no small set of state-controlled developers analogous to the early nuclear weapons programs.

AI instead involves thousands of companies, millions of developers and an enormous private-sector ecosystem, which means trying to force it into the exact structure of the NPT would probably create more confusion than control.

The logic behind non-proliferation remains useful, however, because some capabilities may be too dangerous to distribute without restraint, some deployments may require independent assurance, and some safety requirements may eventually need to operate internationally rather than exist only inside corporate governance frameworks.

A credible regime might therefore include:

1. International scientific assessment of frontier capabilities and risks, including genuine scientific disagreement rather than manufacturing a single official level of concern
2. Mandatory reporting of frontier-scale training and particularly high-risk deployments
3. Independent safety evaluation before release, including testing for autonomy, deception, dangerous cyber capability and loss-of-control indicators
4. Pre-agreed capability thresholds capable of triggering stronger containment, restricted deployment, independent review or temporary pauses when evidence of safety is inadequate
5. Post-deployment monitoring and recovery, including incident reporting, interruption mechanisms and rollback capability
6. Controls around dangerous model-weight releases and high-risk capability transfer
7. Governance of frontier compute, cloud infrastructure and advanced chips where these materially enable dangerous capability development
8. Independent assurance, so the developer is not the only organisation deciding whether its own product is safe enough to proceed
9. International access and capacity building, because technological colonialism does not become more attractive simply because it arrives with nicer stationery
10. Explicit rules for military AI and critical infrastructure, because pretending these environments sit outside the AI safety problem may be politically convenient but is a poor control strategy

The real lesson

The atomic bomb forced governments to confront something that technology has repeatedly demonstrated: invention and governance rarely arrive together, and governance usually follows after the technology has already altered the strategic environment it was intended to manage.

Artificial intelligence is not the atomic bomb. It is broader, faster, more commercial, easier to diffuse and much more deeply integrated into ordinary life, which arguably makes the governance challenge harder rather than easier.

The useful lesson from nuclear non-proliferation is therefore not that AI should be governed exactly like nuclear weapons, but that strategically important technologies eventually reach a point where voluntary restraint and technological monopoly are no longer credible control mechanisms. When that point arrives, governance must adapt by developing shared norms, verifiable obligations, credible assurance, meaningful transparency, and mechanisms capable of restricting the most dangerous uses without eliminating technical benefits.

The difficulty is that this must happen while the technology itself continues to move.

The risk is not simply that AI becomes extremely powerful, but that capability grows faster than the institutions responsible for understanding and controlling it. The September 2026 warnings add another dimension because capability may advance faster than developers can demonstrate that they still have reliable control over the systems they are creating.

Experts can reasonably disagree about the probability of catastrophic outcomes without that disagreement rendering the problem irrelevant. Risk management has never required perfect foresight; it requires making decisions under uncertainty, using the evidence available and taking both probability and consequence seriously.

That does not mean treating every apocalyptic prediction as established fact. A mature governance regime would distinguish between harms that are already demonstrated, high-impact scenarios that are plausible but uncertain, and claims that remain speculative, then apply controls proportionate to each category rather than pretending that every risk deserves the same response.

This is where the nuclear analogy still has real value because it reminds us that safety cannot depend indefinitely on optimism, secrecy or the assumption that the organisations moving fastest will always know exactly when they should stop. Nuclear history suggests that once strategically important technology begins to diffuse, governance becomes progressively harder, which means the most useful question is no longer whether regulation should begin before or after proliferation.

The opportunity is already disappearing.

The question now is what kind of governance can still be built while diffusion is underway.

 

 

References

[1] United Nations Audiovisual Library of International Law (n.d.). Treaty on the Non-Proliferation of Nuclear Weapons. United Nations Office of Legal Affairs. Available at: https://legal.un.org/avl/ha/tnpt/tnpt.html

[2] United Nations Office for Disarmament Affairs (2022). Treaty on the Non-Proliferation of Nuclear Weapons: Tenth Review Conference. United Nations. Available at: https://meetings.unoda.org/npt-revcon/treaty-non-proliferation-nuclear-weapons-tenth-review-conference-2022

[3] Stanford Institute for Human-Centered Artificial Intelligence (2025). The 2025 AI Index Report. Stanford University. Available at: https://hai.stanford.edu/ai-index/2025-ai-index-report

[4] International AI Safety Report (2026). 2026 Report: Extended Summary for Policymakers. Available at: https://internationalaisafetyreport.org/publication/2026-report-extended-summary-policymakers

[5] International Atomic Energy Agency (n.d.). Treaty on the Non-Proliferation of Nuclear Weapons: The Full Text. Available at: https://www.iaea.org/sites/default/files/publications/magazines/bulletin/bull10-4/10403501117.pdf

[6] International Atomic Energy Agency (n.d.). Additional Protocol for Verification of Nuclear Safeguards. Available at: https://www.iaea.org/topics/additional-protocol

[7] Comprehensive Nuclear-Test-Ban Treaty Organization (n.d.). The Comprehensive Nuclear-Test-Ban Treaty. Available at: https://www.ctbto.org/our-mission/the-treaty

[8] Federation of American Scientists (2026). Status of World Nuclear Forces. Available at: https://fas.org/initiative/status-world-nuclear-forces/

[9] Arms Control Association (2026). New START Expires as U.S. Urges ‘Modernized’ Treaty. Available at: https://www.armscontrol.org/act/2026-03/news/new-start-expires-us-urges-modernized-treaty

[10] United Nations High-level Advisory Body on Artificial Intelligence (2024). Governing AI for Humanity: Final Report. United Nations. Available at: https://digitallibrary.un.org/record/4062495

[11] International AI Safety Report (2026). International AI Safety Report 2026: Extended Summary for Policymakers. Available at: https://internationalaisafetyreport.org/sites/default/files/2026-02/ai-safety-report-2026-extended-summary-for-policymakers.pdf

[12] United Nations General Assembly (2025). A/RES/79/325: Terms of Reference and Modalities for the Independent International Scientific Panel on Artificial Intelligence and the Global Dialogue on Artificial Intelligence Governance. Available at: https://digitallibrary.un.org/record/4087699/files/A_RES_79_325-EN.pdf

[13] Organisation for Economic Co-operation and Development (2024). OECD AI Principles. Available at: https://www.oecd.org/en/topics/ai-principles.html

[14] European Commission (2024). AI Act Enters into Force. 1 August 2024. Available at: https://commission.europa.eu/news-and-media/news/ai-act-enters-force-2024-08-01_en

[15] Council of Europe (2024). Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law. Available at: https://edoc.coe.int/en/artificial-intelligence/11926-council-of-europe-framework-convention-on-artificial-intelligence-and-human-rights-democracy-and-the-rule-of-law.html

[16] National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). Available at: https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10

[17] UK Government (2023). The Bletchley Declaration by Countries Attending the AI Safety Summit, 1-2 November 2023. Available at: https://www.gov.uk/government/publications/ai-safety-summit-2023-the-bletchley-declaration/the-bletchley-declaration-by-countries-attending-the-ai-safety-summit-1-2-november-2023

[18] G7 Hiroshima Process (2023). International Code of Conduct for Organizations Developing Advanced AI Systems. Available at: https://www.mofa.go.jp/files/100573473.pdf

[19] Anthropic (2026). Assessing Claude Mythos Preview’s Cybersecurity Capabilities. 7 April 2026. Available at: https://www.anthropic.com/research/mythos-preview

[20] Anthropic (2026). Project Glasswing: An Initial Update. 22 May 2026. Available at: https://www.anthropic.com/research/glasswing-initial-update

[21] RAND Corporation (2025). Full Stack: China’s Evolving Industrial Policy for AI. Available at: https://www.rand.org/pubs/perspectives/PEA4012-1.html

[22] Atalan, Y. (2026). What to Know About Chinese AI Models. Center for Strategic and International Studies, 2 July 2026. Available at: https://www.csis.org/analysis/what-know-about-chinese-ai-models

[23] National Institute of Standards and Technology, Center for AI Standards and Innovation (2026). CAISI Evaluation of DeepSeek V4 Pro. 1 May 2026. Available at: https://www.nist.gov/news-events/news/2026/05/caisi-evaluation-deepseek-v4-pro

[24] Anthropic (2026). 2028: Two Scenarios for Global AI Leadership. 14 May 2026. Available at: https://www.anthropic.com/research/2028-ai-leadership

[25] U.S. Department of Defense (2025). Military and Security Developments Involving the People’s Republic of China 2025: Annual Report to Congress. Available at: https://media.defense.gov/2025/Dec/23/2003849070/-1/-1/1/ANNUAL-REPORT-TO-CONGRESS-MILITARY-AND-SECURITY-DEVELOPMENTS-INVOLVING-THE-PEOPLES-REPUBLIC-OF-CHINA-2025.PDF

[26] Associated Press (2026). Trump Administration Lifts Restrictions on Anthropic’s Claude Models After Cybersecurity Alarm. 1 July 2026. Available at: https://apnews.com/article/028db5135128fce6b38c873bf9cb5e09

[27] The Guardian (2026). AI Could Kill All Humans in Next Decade, Warn Experts: But How Seriously Should We Take Them? 9 September 2026. Available at: https://www.theguardian.com/technology/2026/sep/09/ai-superintelligence-risks-warnings-scientists-politicians

[28] OpenAI (2026). Paul Christiano Joins OpenAI Foundation Board. 9 September 2026. Available at: https://openai.com/index/paul-christiano-joins-openai-foundation-board/

[29] Anthropic (2026). An Alignment Assessment of Recent Cybersecurity Incidents. 9 September 2026. Available at: https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents