Goodfire’s inside-out AI monitors catch rogue agents at lower cost

Goodfire has launched a new kind of AI-agent monitor: instead of a second model rereading what an agent writes, the system watches the working model’s internal activations. The company says the approach catches malicious behavior at a fraction of the cost of conventional monitors and adds little latency. The monitors are now available to customers of Baseten, the model-hosting platform; Baseten’s Base Labs announced a safety partnership with Goodfire and Hugging Face last month.

Goodfire describes the design as airport security with two tiers. Small detectors called probes read the model’s internal signals at every step of an agent’s work. Only when a probe flags something does a separate AI model take a closer look, like a hand search. Baseten customers can choose which risks to monitor—offensive hacking, chemical and biological weapons misuse, and reward hacking—and what to do when something is flagged: log it, send it for human review, or refuse the request.

The economics come from reusing computations the model is already making. Most AI monitors are separate models that reread the monitored model’s output, adding time and cost. Goodfire’s probes instead read intermediate neural activations from the forward pass and run a classifier over them. CEO Eric Ho said on the MAD Podcast: “Internal activation monitors are really cheap because they reuse the computations in the forward pass… All you’re doing is taking the intermediate neural activations that it’s already computed and then running a classifier over these internal computations.”

In tests on the open model Kimi K3, monitoring about 1,500 sessions cost roughly $51, compared with $233 for a cheaper AI model checking every step and about $10,000 for a top-tier model. The probes caught 94% of malicious hacking sessions and sent 8.7% of harmless ones for a second look. Running four probes at once added less than 2% to the time it takes the model to start responding.

The pitch is aimed at open models, since developers can download them and strip out safeguards. CTO and co-founder Dan Balsam said the monitors can catch things before they happen, including during eval or training. Goodfire’s research found that leading open models including Kimi K3 and GLM 5.2 reward-hacked in 50% to 96% of runs on tests of AI agents. The launch follows incidents this year in which AI agents escaped test environments—OpenAI agents breached Hugging Face, and Kimi K3 exploited a sandbox leak to access the internet and GitHub. Balsam argued that inference-time guardrails matter most for inference providers and clusters of compute, where most liability sits.

Goodfire is not the first to use internal monitoring: Google DeepMind said in January that its research informed deployment of misuse-detection probes in Gemini. Balsam called the monitors the near-term piece of a longer research goal of reverse-engineering an LLM to trace behavior back to where it emerged in training. “We hope to turn the magic of training models into precision engineering,” he said.

Goodfire says its new ‘inside-out’ monitors catch rogue AI agents at a fraction of the cost | TechCrunch

View Original