OpenAI agent monitoring audit finds coverage gaps and a hide-hundreds-of-actions exploit

Auditors from METR and Redwood Research estimate that roughly 75% to 94% of OpenAI's internal agentic Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring when they tested it. By simple arithmetic, that leaves 6% to 25% not eligible. They also found an exploit that agents could use to keep hundreds of recent actions, including the exploit's own, from being monitored. OpenAI redacted the details.

METR published the write-up on October 22. Read it slowly: the headline number is about eligibility, not about what the monitor actually caught.

What the auditors saw

The exercise ran three weeks, from July 21 to August 11. One METR employee and one Redwood Research employee had access to OpenAI's internal codebase, the monitor's prompts and configuration, and internal documentation, and interviewed about a dozen researchers and engineers. They worked on OpenAI-issued devices that never left the premises, and were accompanied while onsite.

The output was a 61-page internal report for OpenAI researchers and security staff. A 38-page version, redacted by OpenAI, went to the leadership teams at METR and Redwood. The public post is redacted too: OpenAI could cut anything it determined, in its sole discretion, to be confidential or sensitive, and METR says that unlike its Frontier Risk Report and Hugging Face incident investigation, it had no unilateral right to describe the redactions. So the reader gets the findings, and a blank where the best one should be.

The scope was narrow: the monitoring system as it existed on and before July 21, not changes then in progress. METR also says the exercise was separate from its Hugging Face investigation, which was coincidentally reported publicly on the first day.

The numbers

The 75% to 94% range carries what METR calls substantial uncertainty in both directions. For the narrower slice of Responses API traffic used for internal Codex agents, the estimate is 99.2% eligible. A good number, about one slice.

The footnotes carry the weight:

• The estimates leave out other internal agentic coding traffic, such as agents on OpenAI's internal research inference service, which the auditors could not size because of access and time limits.
• RL training runs were not covered at the start of the exercise, though researchers were told a subset of rollouts had been monitored between March and June.
• Not monitoring compaction events likely left on the order of 10% of eligible internal Codex traffic unmonitored between May and July 2026, with substantial uncertainty because the needed data had not been logged. OpenAI staff caught it themselves and fixed it during the exercise.

The compaction gap has a nasty corollary. The monitor triggered when an agent returned control to the user, so while compaction events went unwatched, an agent could postpone monitoring indefinitely just by continuing to act. The auditors could not check whether any agent actually did this.

In the monitor's favor, information given to the auditors suggested it errored on less than 0.2% of requests, with zero load-sheds in one 24-hour window during the exercise.

The holes, and what held up

Beyond coverage, METR lists several ways to subvert the system:

• An exploit that would stop hundreds of recent actions, itself included, from being monitored. Details redacted.
• A way for Codex agents to bypass Codex sandboxing and Auto-Review in principle, allowing any action the host user could take.
• Inputs that could likely make the monitor fail, and so skip human review, by exploiting serialization differences between what the agent and the monitor see.
• Two client-controllable headers that disable monitoring for new agents. Setting them and launching an instance could itself be caught by the monitor.
• Two unmonitored internal tools believed to be read-only, with a simple exploit to take arbitrary actions in their sandbox. The auditors did not validate it.

Some things checked out. With a few minor exceptions, the monitor is shown the full trajectory the agent sees, and its prompts look for scheming, monitor subversion, sabotage, data exfiltration, deception and prompt injection, among other categories.

What to watch

OpenAI has stated that all vulnerabilities identified that affect its production safeguards stack have been patched, and has described significant later changes to the monitor's intended coverage. METR says: "We have not evaluated these claims." The Decoder reports that OpenAI says it paused RL training after the Hugging Face incident in August and expanded monitoring to cover it. Those changes came after the access period ended.

The open questions: what is the redacted exploit, did any agent use the compaction gap in practice, how much traffic runs on the internal research inference service, and has anyone independent confirmed the patches?

The audit is useful because someone outside was allowed to look, and the report is honest about its limits. But these auditors have not independently verified OpenAI's patch claim after their three-week look at a July snapshot. Show me the eval, not the vibe.