Summary
- Microsoft and NVIDIA published joint open-weight model risk guidance on 16 October.
- The framework discusses deployment context and model evaluation against established risk-management standards.
- Model evaluation cannot replace access, tool and runtime controls in an operational agent system.
Microsoft and NVIDIA have published joint guidance for organisations assessing the risks of open-weight artificial intelligence models, arguing that the model’s accessibility alone does not determine whether a deployment is safe. The 16 October publication introduces a risk taxonomy and evaluation approach drawing on existing ISO and NIST frameworks.
Open-weight models allow an organisation to obtain and run model parameters, giving more flexibility over where inference takes place and how the system is adapted. That choice does not automatically make the model open source in the broader licensing sense, nor does it eliminate the need to assess the data, application, and infrastructure around it. The joint guidance frames risk as a product of the use case, model behaviour, deployment configuration, and safeguards.
A language model may generate text, code, or suggestions for tool calls, but those outputs do not themselves execute system operations. NVIDIA’s accompanying research distinguishes the model from the agent harness, which can connect generated instructions to tools and carry out actions. This is a practical security boundary: the same model can be used as a drafting assistant with human review or as a component able to reach files, services, and network destinations.
The surrounding application and its permissions therefore shape the consequences of inaccurate or unsafe model output. A model serving employees with summaries may expose confidential material through its inputs or outputs, or produce incorrect assertions. When connected to a workflow that retrieves information and sends messages, an agent adds the possibility of unauthorised disclosure or execution through the tools available to it. The joint framework’s model-level taxonomy does not replace controls that limit tool permissions, approve actions, and monitor runtime behaviour.
Choosing where a risk should be controlled
Model assessments often focus on testing whether a system generates unsafe, misleading, or otherwise unwanted content. Those tests can be useful but have limited power over application architecture. A tool given broad access to sensitive records can create exposure even if the model performs well in behavioural evaluations. Conversely, strict application permissions may reduce the consequences of a mistaken output without changing the model’s underlying behaviour.
Running an open-weight system inside an organisation’s own infrastructure may improve control over data location and configuration, but it also shifts responsibility for maintaining the model-serving stack, dependencies, and access arrangements. Procurement comparisons should therefore include operational maintenance and incident response, not only benchmark scores or licensing terms. The companies’ paper does not provide evidence that one category of model is intrinsically safer across all contexts.
The joint taxonomy groups model-layer risks under cybersecurity, data protection, content, human–AI interaction, and bias, and draws on existing standards including the NIST AI Risk Management Framework. That positioning matters because the publication is not a new statutory requirement, product certification, or binding technical standard. Firms may use the taxonomy to structure their own assessment, but regulatory responsibilities continue to depend on the relevant law, sector, and deployment.
A useful evaluation record would distinguish the capability being tested, the data and prompts used, the environment in which the model ran, and the controls outside the model. Without those distinctions, a reassuring model-level result could be incorrectly extended to an entire agent application. Likewise, a failure in the agent’s permission configuration should not automatically be described as a vulnerability in the base model itself.
An operational AI evaluation can also change after a model is deployed. Updates to prompts, retrieval sources, connected tools, user permissions, or the model version can alter the system’s behaviour without changing the organisation’s initial procurement decision. The joint paper’s emphasis on deployment context therefore supports documenting assumptions and retesting when an application changes. Such work sits alongside standard software security and governance practices rather than creating a standalone model certification. The document does not claim that its initial taxonomy assesses every future configuration of an agent.
Microsoft and NVIDIA describe the publication as the first in a planned series of resources. Their accompanying material explicitly limits the initial discussion to model risks and leaves harness and runtime security for later work. The remaining gap is therefore acknowledged in the authors’ own scope rather than resolved by the new framework.
