Everyday wet-lab science is messy: experiments fail, images are ambiguous, and observations conflict. To help scientists accelerate biological research, AI agents need to interpret the raw experimental evidence used in wet labs. Existing benchmarks built around charts and images selected for scientific papers, according to the post, miss this ambiguity and the troubleshooting it requires. That gap motivates BioBasicsBench, a new benchmark developed with Ligo, a biotech company focused on enzyme design. Its 100 tasks test whether agents can interpret raw experimental data, identify errors, and recognize when the evidence cannot support a conclusion. Misreading an image or overlooking a problem with controls can send scientists toward wrong conclusions and wasted experiments.

The tasks span five areas of routine wet-lab work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. Inputs preserve the forms of evidence scientists work with: gel and blot images, plate photographs, microscopy stacks, sequencing reads, plate-reader exports, videos of bench work, and reference sheets such as ladder charts and plasmid maps. Scientists wrote tasks using data from their own experiments, pairing each question with expert-written grading criteria. Each task went through an internal review by a Ligo scientist who had not written it, an external review by a PhD-level wet-lab scientist, and a final internal review. Agents return structured answers with the requested conclusion, measurement or recommendation and, where required, supporting reasoning. Credit is assigned against individual rubric criteria, so the benchmark can distinguish a correct conclusion from a partially correct analysis. Most tasks test several skills together: biological knowledge, interpretation of images and other data, error detection, and protocol understanding.

The evaluation covered seven models—GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna, and GPT-6 Astra—across 37 model and reasoning-effort settings, with five attempts on each of the 100 tasks per setting, for a total of 18,500 results. Mean reward is reported as the average task score as a percentage of available credit. At the xhigh setting, GPT-6 Astra earned 66.7%, compared with 43.8% for GPT-5.6 Sol. Astra also improved substantially at low reasoning effort, which the authors suggest points to stronger native multimodal capabilities as a key factor for everyday wet-lab analysis.

Astra showed qualitative gains in recognizing what data can and cannot show. In one example, GPT-5.6 Sol classified cells as dead based on brightness from a dead-cell dye, while Astra recognized that a second dye could contribute to the same signal and called for better controls. In a cloning experiment, GPT-5.6 Sol based conclusions on the negative control plate with no colonies; Astra noticed the plate was labeled with the wrong antibiotic and rejected it. In a microscope series, GPT-5.6 Sol looked for a biological explanation for cells apparently disappearing, while Astra correctly recognized a z-stack, a series of images taken at different depths.

The post also documents where Astra still falls short. In a colony PCR task, Astra grouped the colony bands visually correctly but attached reversed experimental identities, selecting the wrong colonies that would lead to purification of the incorrect plasmid. In a microscopy task, it generalized that vacuoles are darker than surrounding cells even though lighter vacuoles were visible in the same image, discounting observations that contradicted its preferred interpretation. On colony counting, its marks failed to resolve close neighbors, missed visible colonies, and counted empty agar, producing an overcount of approximately 12% on the shown plate and inflating the corresponding transformation-efficiency estimate. The authors state plainly that higher benchmark scores do not yet mean models can make reliable judgments in the lab.

BioBasicsBench is intended to make routine work a target for improvement: interpreting raw data, checking controls, and recognizing unsupported conclusions. Progress on these tasks could help scientists spend less time troubleshooting and more time deciding which experiments to pursue. The authors also say tracking these capabilities will help understand their implications for biosafety and biosecurity.