The UK AI Security Institute (AISI) is now using EvalEval’s infrastructure to openly publish evaluation results, a move both organizations say supports more reproducible and verifiable evaluation science. The collaboration is not new: the two groups began joint research at a workshop alongside NeurIPS 2025, and AISI’s feedback helped shape the Every Eval Ever (EEE) schema, the shared reporting format used to describe evaluation runs. This latest phase puts that shared infrastructure into practical use.

The problem the project targets is familiar to anyone who has tried to compare benchmark numbers. As AI deployment accelerates, evaluations are increasingly important evidence about model and system performance, but results are reported across many formats, platforms, and outlets, often without enough information to reproduce them. Re-running the evaluations can itself be prohibitively expensive. EvalEval’s answer has two parts: a shared reporting schema (Every Eval Ever) and an open platform (Evaluation Cards) that bring evaluation results and the information needed to interpret them into a common structure.

This builds naturally on AISI’s existing work to make evaluation more efficient through OptStop, more statistically rigorous through HiBayES, and more standardized in areas including transcript analysis and capability elicitation. Together, the two groups are working to diagnose gaps in evaluation reporting and build shared infrastructure to close them.

What AISI is sharing this time is transcript-level data: publicly reported evaluation methods and findings made available through Evaluation Cards where appropriate. The release includes verified results, context, and configuration information for five benchmarks from the paper’s main experiment—HealthBench, FrontierMath, Humanity’s Last Exam, SWE-Bench Pro, and Terminal-Bench 2.0. It covers six frontier models—Claude Opus 4, Claude Opus 4.5, Claude Opus 4.6, GPT-5, GPT-5.2, and GPT-5.4—plus two related cyber evaluations (Cyber CTFs and The Last Ones), which use a different, partially overlapping set of models. The data accompanies AISI’s paper “How Inference Compute Shapes Frontier LLM Evaluation,” which studies how benchmark performance depends on inference-time compute and evaluation protocol.

A concrete finding illustrates why setup details matter. Performance on Humanity’s Last Exam changes with evaluation protocol and inference compute: each curve shows the cumulative share of attempted tasks solved within a given token count, using the earliest observed success per task. When models received correctness feedback from an oracle after each attempt, they continued to solve additional tasks as token use increased. That behavior is invisible unless the protocol is published alongside the score.

The practical value of open releases is comparability. When results are published with setup information, researchers and practitioners can examine individual studies more closely and compare findings across the wider ecosystem. Where other reports lack these details, releases like AISI’s provide verified reference points for interpreting evaluations in context—for example, by helping researchers understand how setup choices may influence reported performance. As more evaluators adopt EEE, such open comparisons can support broader and more reliable meta-research; the post shows AISI’s Terminal-Bench 2.0 results alongside other reported evaluations for the same models under different setups.

The announcement ends with an explicit call for participation. Model developers are asked to report verified evaluation results; evaluation developers to report benchmarks and run data using the Every Eval Ever schema; and evaluation, governance, and policy researchers to explore Evaluation Cards by benchmark or model, or use it to examine the state of evaluation reporting as a whole. EvalEval describes itself as a research community whose goal is to improve evaluation science, address the lack of consensus around documenting evaluation applicability and utility, and broaden coverage of the impacts that matter for scientific research and policy analysis. Its flagship projects are Every Eval Ever and Evaluation Cards, with the latter combining benchmark metadata, evaluation-run data, and model metadata into interpretable records that make it easier to tell when apparently similar scores were produced under meaningfully different conditions. AISI, for its part, is described as a research organisation within the UK government’s Department for Science, Innovation and Technology, with a mission to equip governments with a scientific understanding of advanced AI risks.