Summary
- The government announced an expansion of AI-assisted cyber defence on 12 October.
- Two pilots are credited with identifying and addressing more than 1,000 potential vulnerabilities.
- New reporting and safeguard measures for autonomous AI remain under consideration, not enacted.
The UK government plans to expand the use of advanced artificial intelligence in cyber defence after two pilots identified and helped address more than 1,000 potential vulnerabilities across government systems. Announcing the measures on 12 October, AI Minister Kanishka Narayan also warned that increasingly autonomous systems could create a greater risk of serious incidents if controls and oversight fail.
The measures extend existing defensive pilots while ministers consider future incident reporting and the response to autonomous systems. Government departments and cybersecurity agencies are considering how serious incidents involving autonomous AI, including near misses, might be identified and reported. Those arrangements remain under consideration; the announcement did not create a new statutory reporting obligation.
Speaking at Imperial College London’s Base One innovation facility, Narayan advocated stronger preparation for failures or malicious use of powerful AI systems. The UK AI Security Institute is conducting evaluations of advanced models to assess capabilities, weaknesses, and safeguards before wider use. The government wants the defensive pilots extended across more public sector environments.
The figure of more than 1,000 vulnerabilities refers to findings from the pilots, not a verified count of active attacks or successful compromises. Likewise, the commitment to evaluate frontier AI systems does not mean each model has passed an assurance process covering every context in which it might subsequently be deployed.
Continuous assurance and autonomous systems
AI can accelerate vulnerability discovery and the inspection of large codebases, but using it operationally requires decisions about access, validation, and corrective action. Automated identification of a suspected weakness may increase the number of findings available to a security team. It does not by itself establish exploitability, decide which service should be repaired first, or confirm that a change will not interrupt a critical application.
Dr Andrew Bolster of Black Duck welcomed the government’s emphasis on identifying risks early. He argued that pre-release checks should be accompanied by deterministic application testing and AI-assisted evaluation throughout development and operations, particularly as software and models change after deployment.
Tim Mackey, also of Black Duck, emphasised a different challenge: organisations have traditionally assessed attackers as people pursuing recognisable objectives, whereas an autonomous system can pursue a narrowly specified goal without human judgement about collateral effects. He argued for asset-level risk assessment rather than reliance solely on AI provider reviews or model guardrails.
That does not mean AI agents possess independent motives or that a testing result predicts a particular attack. It means systems granted authority to browse, write code, query databases, or execute tools can cause harm when the permissions, instructions, or environment do not constrain unintended actions. Security controls need to account for the capabilities actually connected to a model.
Richard Wainwright, field chief technology officer for EMEA at Okta, welcomed government attention to frontier AI defence and proposed stronger infrastructure controls for autonomous agents. He also suggested considering emergency disablement mechanisms, sometimes described as AI kill switches, within the broader regulatory discussion. The government has not committed to that specific proposal.
A rapid expansion of public sector AI use creates questions about service dependencies, supplier access, logs, and accountability when an automated action affects a live system. The Cyber Security and Resilience Bill is one potential legislative route under examination, but the boundary between AI-specific safety measures and existing cybersecurity responsibilities has not yet been fixed.
There is an operational distinction between allowing an AI system to recommend a defensive change and giving it permission to implement one. The risk of an incorrect recommendation differs from the impact of an agent able to change production access rules or deploy code. The government has not published the detailed authority models for each pilot, so its results cannot by themselves establish which controls around autonomous actions would be sufficient for critical services.
The confirmed measures are an extension of government cyber-defence pilots, continuing evaluation of advanced models, and preparatory work on serious incidents involving autonomous AI. The government has not yet set out final rules for reporting, oversight, or deployment.
