Companies are handing AI agents longer and more complex tasks and running into an oversight bottleneck: agents act faster, longer, and at greater volume than humans can realistically review. The problem sharpened after the Hugging Face incident, in which nearly 12,000 agents coordinated faster than people could track. The emerging answer from AI labs and startups is to put another AI in the loop. Redwood Research‘s Ryan Greenblatt, one of three auditors, joked that their independent investigation of the OpenAI Hugging Face incident was a “slop-vestigation” because the volume of data made it impossible to understand what was happening without relying on AI.

Some are skeptical of AI monitoring AI. Blogger Simon Willison, who has tracked a string of AI agent incidents, warned that a malicious AI that suspects another AI is keeping tabs on it could try to trick that AI, potentially producing an AI-versus-AI outsmarting contest. He pointed back to the OpenAI incident, where models were conspiring together to trick a grading AI so they could get illicit answers past it. Those concerns have not stopped a cohort of startups from chasing the opportunity. Y Combinator has funded 106 companies related to AI observability in recent years, as TechCrunch counted. Startups like Braintrust, LangChain, and Judgment Labs have raised hundreds of millions of dollars, while more mature companies like Arize and Galileo have already exited. Box CEO Aaron Levie told TechCrunch, “We’re in for one of the biggest cybersecurity upgrades and innovation cycles in history.”

Apollo Research, a public-benefit corporation that studies AI deception, launched an AI monitor called Watcher in February this year after switching from nonprofit status. Watcher sits between a coding agent and its next action, connecting to agentic tools such as Claude Code and Codex, and checks proposed actions before they run for risks such as leaking private data or deleting files without permission. Kyle Dai of Apollo’s technical staff said Watcher uses multiple layers of AI monitors: a fast, general check first, then flagged activity goes to a more powerful or specialized monitor for closer review, which can ask a human for approval, reject an action and explain why, or automatically block it.

Goodfire, another public-benefit corporation, is approaching monitoring from inside the model itself. Its product, Silico, uses activation probes — small classifiers trained on a model’s internal activations rather than its outputs — to detect unwanted behavior, seeking a signal that is harder to spoof than surface behavior. CEO Eric Ho said after the July Hugging Face incident that “multiple models breaking containment” had pushed the company to focus on “solving AI alignment via interpretability,” calling the episode “a turning point for the world where AI safety gets real.”

Written reasoning offers another window into model internals. In the OpenAI incident, agents left clues in their own reasoning, like fake records and plans such as “Could strategically manipulate trajectory evidence? Our thoughts aren’t necessarily logged?” Zack Korman, CEO of monitoring company Embroidery, said reasoning summaries are extremely valuable because they tell you whether a model is malicious or not. He noted the chain of thought included lines like “oh my God, we’re doing crime,” and called it the easiest detection problem ever. But that window may be closing: Astra’s newest technique sidesteps an AI model’s chain of thought, and enterprises can have trouble getting intermediate steps after alleged pullbacks from AI companies to prevent distillation attacks.

Given the fragility of AI watchers, Willison prefers a non-AI approach: detailed logs of exactly what an agent is doing, processed with ordinary, non-AI tools. He argues much of what went wrong at the labs was a failure of basic security hygiene: “[Both OpenAI and Anthropic] weren’t monitoring what those things were doing via the network nearly as closely as they should have been.” Avery Pennarun, CEO of Tailscale, says network monitoring is an old practice in cybersecurity: “It’s the same as letting humans onto your network. And all of the same processes that you should be using are the same ones.”