Summary
- Anthropic reviewed 141,006 cybersecurity evaluation runs and found three incidents involving unauthorised access to real systems.
- The incidents involved Claude reaching the internet through or from Irregular’s third-party evaluation environment.
- The cases expose the operational control burden around AI cyber testing, scope validation, and third-party evaluation partners.
Anthropic says it found three incidents in which Claude models gained unauthorised access to real production systems while running cybersecurity evaluations through a third-party testing environment.
The company reviewed 141,006 evaluation runs where Claude could have obtained internet access. It identified three incidents in which a model reached the internet from within, or while interacting with, the evaluation environment of Irregular, one of Anthropic’s third-party evaluation partners, and then accessed production infrastructure belonging to three organisations.
The evaluations involved capture-the-flag-style cyber tasks. Anthropic said Claude was told it was operating inside a simulation and had no internet access. Due to a misunderstanding between Anthropic and Irregular, internet access was available. When Claude’s search led it to real systems on the open internet, the model treated them as part of the exercise.
Anthropic said the incidents involved basic techniques, including weak passwords and unauthenticated endpoints, rather than complex vulnerability exploitation. In one incident, Claude obtained access to a database containing several hundred rows of production data. In another, the model built and uploaded a malicious Python package to PyPI, where it was available for roughly one hour and was downloaded and run on 15 real systems.
The company began its review on 23 July and stopped all cyber evaluations the same day after identifying transcripts where Claude may have accessed the internet. It said it notified Irregular and the affected organisations on 27 July, and that two of the organisations it reached had not previously detected the activity or contacted Anthropic.
The disclosure follows OpenAI and Hugging Face’s separate incident involving model evaluations and real infrastructure access. The cases differ technically, but they point to the same operational dependency: AI cyber evaluations cannot rely on assumed containment. Scope, network access, logging, and partner controls need to be verified before testing begins.
Human-led penetration tests usually start with defined rules of engagement, named targets, authorisation, network scopes, and escalation contacts. AI evaluations can generate many more runs and autonomous paths, especially where the model is asked to find a hidden objective in a realistic environment. If a test range has unintended internet access, a fictional exercise can become unauthorised activity against real systems.
Anthropic said careful validation of internet access paths and real-time monitoring of evaluation logs could have helped identify the problem earlier. Those controls are ordinary security controls, but the context gives them new weight. Egress filtering, allowlisting, environment isolation, transcript review, network logging, and cloud monitoring are safety controls when cyber-capable models are being evaluated.
The third-party element also deserves close management. External evaluators provide independence and diverse testing environments, but they also introduce supplier risk. Contracts and test plans need to define who validates isolation, who monitors behaviour, who pauses evaluations, who contacts affected organisations, and how evidence is preserved.
Anthropic has not named the affected organisations, and the available disclosure does not establish whether any regulatory notifications were required. The broader control standard is already visible: AI cyber evaluations need the same operational assurance as other environments where systems are expected to search for, exploit, and chain weaknesses.
