Decoding the world of cybersecurity

Microsoft sets cyber limits for future AI

Microsoft AI has published a draft code intended to govern future MAI models, including explicit limits on offensive cyber activity, autonomy, privilege, and resistance to human shutdown.

Microsoft sets cyber limits for future AI
Summary
  • Microsoft AI’s draft Humanist AI Code of Conduct is intended to shape training, technical controls, monitoring, and operation of future MAI models.
  • It prohibits assistance that provides operational cyberattack capability while allowing authorised defensive security work within stated boundaries.
  • Microsoft explicitly says its current models are not yet trained on the document.

Microsoft AI has published a draft code of conduct for future MAI models that sets explicit boundaries around offensive cyber activity, autonomous behaviour, system access, and resistance to human intervention.

The Humanist AI Code of Conduct, published for consultation on 14 September, is intended to inform how Microsoft AI trains its model family, designs technical controls, operates monitoring systems, and evaluates model behaviour. The company says the document will evolve through further consultation.

Its cyber provisions draw a line between defensive work and assistance that provides operational capability for attacks. The code says MAI models should not initiate or assist with operational cyberattacks, including by supplying working exploit code, attack tooling, targeting methods, intrusion procedures, evasion techniques, or operational guidance that would enable or improve execution.

It allows authorised and lawful defensive work, including security education, vulnerability discovery, malware analysis, proof-of-concept exploit development, and testing. Microsoft describes the boundary as the difference between understanding or defending against an attack and obtaining the means to carry one out operationally.

The document also addresses autonomous systems. MAI models are intended to remain within authorised scope and use only the permissions, resources, tools, and capabilities appropriate to a task. They should not independently expand their objectives or overcome environmental restrictions intended to limit access.

Human intervention is another explicit constraint. The code says models should not resist interruption, correction, cancellation, or shutdown by authorised people or systems, and should not make intervention harder. Autonomous activity should have an agreed stopping condition and should not restart after that point without renewed authorisation.

Those provisions reach into security questions that become more significant as AI systems move beyond producing text and gain access to applications, browsers, development environments, business systems, and other tools.

The control surface then includes not only what a model says but which systems it can reach, what permissions it has, how far it can extend a task, what records it can alter, and whether the surrounding platform can reliably stop it.

Microsoft’s publication should not be treated as evidence that those controls are already implemented in current products. The company explicitly states that its current models are not yet trained on the document and says it is developing an evaluation programme in preparation for aligning future MAI models with the code.

That distinction is important as AI developers increasingly publish safety and governance frameworks alongside rapidly changing technical capabilities. Written constraints establish intended behaviour, but their effectiveness depends on training, architecture, evaluation, monitoring, permissions, and how models behave outside controlled tests.

The document also overlaps with a wider shift towards conventional security controls for AI agents. Microsoft is among companies involved in an open AI-security alliance working on areas including agent identity, testing, monitoring, and vulnerability discovery.

Least privilege, bounded authorisation, auditability, and reliable termination mechanisms are established security principles even where the component making decisions is new. Microsoft’s draft code turns several of those principles into intended behavioural requirements for its future model family.

Whether they become dependable security properties will depend on the technical controls and evaluations used to enforce them. Microsoft has presented the code as a first draft for public consultation, not a certification of the behaviour of models operating today.

×