AI labs have staked their reputations on model benchmarking, the supposedly neutral and accurate system of evaluations meant to show whether one product is better than another. But according to Hugging Face chief policy officer Irene Solaiman, speaking at TechCrunch Disrupt, that system is not yet stable enough to be trusted. She described evaluation methods as an evolving science, one the industry needs to make more consistent before it can rely on them.

Solaiman put trust partly in reproducibility: can an evaluation result be reproduced, and does the evaluator have the level of model access needed to verify it? That is where open weights matter in a big way, she said — they make the model itself available for inspection, which is much harder with a closed system. Reproducibility has long been a sticky aspect of benchmarking, and it remains a topic of significant industry debate.

There is also the question of how evaluations compare with each other. Solaiman said they become ineffective relatively quickly and the ecosystem around them is not as thriving as it needs to be. She pointed to a growing hunger for an embedded evaluator ecosystem — neutral third-party observers placed inside AI organizations — but said the qualifications that determine who can effectively evaluate what tasks still need a lot more standardization. Labs have pushed for this third-party approach instead of active government regulation, in part because of safety incidents; Hugging Face itself was famously the target of one such incident. Yet doubts remain about how neutral and effective embedded evaluators will be. The article cites the backlash to Anthropic‘s decision to hire Accenture as its first evaluator, with critics mocking the idea that a consulting firm was qualified for the task.

Solaiman also described a shift in evaluation scope. Evaluations are moving from testing just the model to testing the entire system around the model, including the harness — the tool a user is using to access the model — and the agent. She noted that the harness has a lot to do with performance: a weaker model with a better harness could easily outperform a frontier model. But while model evaluations are plentiful, harness evaluations are not, and better evaluations for the holistic agent are still being developed. Agentic evals, she said, are really hard and really expensive, even though that is where many people are getting value.

The broader message is that the evaluation space is currently a bit of a free-for-all. If benchmarking is to remain a useful tool for industry leaders, Solaiman argued, the field may need a more uniform, industry-wide approach where everyone is on the same page.