This follow-up post describes ThinkingBox-Training, the new open-source repository that turns database state checks into reinforcement learning rewards for agent post-training. ThinkingBox was originally built as a benchmark grader: it compares the final database state with the expected outcome and returns an executable 1/0 verdict. The authors realized the same verdict could act as a reward signal, so they fine-tuned Qwen3.8-27B on training tasks outside ThinkingBox-Bench using terminal-state rewards. The post is a joint Microsoft and Hugging Face blog, and the training code is released in microsoft/thinkingbox-training.

The reported results come from 50 steps of GRPO on a curated training pool. On the sealed 507-task ThinkingBox-Bench benchmark, pass@1 rose from 51.70% to 60.78%, while observed 20/20, the share of tasks that passed all twenty recorded attempts, rose from 38 tasks (7.50%) to 107 tasks (21.10%). Pass@20, tasks solved at least once, declined slightly from 453 to 446. The authors interpret this as sharpening: the model became more repeatable rather than broader. Across the five domains, pass@1 and observed 20/20 rose everywhere, while pass@20 was mixed; retail gained the most on observed 20/20, and auto insurance had the largest breadth loss. Cost per dependable task, estimated from the full 507 x 20 campaign, fell from $24.36 to $8.00, and cost per successful attempt fell from $0.177 to $0.139. The dollar figures are described as a comparative serving index, not invoices or cloud bills.

The training loop treats every rollout as a complete episode with a fresh backend, an isolated MCP session, and GPT-5 Chat as the simulated user. Rewards come mostly from deterministic executable checks of terminal state; a minority of narrow rubric requirements invoke the configured LLM judge. Training and evaluation must parse Qwen output identically, including reasoning extraction, tool calls, and argument typing; the shared interpretation code lives in trainer/core/tb_interpret.py. The reference run used full-parameter fine-tuning on 187 tasks with token-mean GRPO, 18 groups of 8 rollouts per update, 24 H100s, AdamW with a 1e-6 learning rate and zero weight decay, and 50 training steps. Zero-variance groups were refilled DAPO-style rather than scored as failures.

Task selection relies on what the authors call offline policy-relative difficulty. Candidate tasks are probed repeatedly with the base policy, and each task’s latent success probability is modeled with a Jeffreys posterior, Beta(k + 1/2, n – k + 1/2). Tasks are bucketed by posterior mass with boundaries at 0.08, 0.40, 0.60, and 0.90: rarely solved tasks and nearly always solved tasks are dropped, while difficult, intermediate, and frequently solved tasks are kept. The reasoning is that all-fail or all-pass groups produce no GRPO contrast, whereas mixed groups supply the reward difference the algorithm can learn from. One example task succeeded on 3 of 8 probes, producing a Beta(3.5, 5.5) posterior with 54.0% mass in the Difficult band, and later produced the mixed reward group [0, 0, 0, 0, 1, 1, 1, 1]. No ThinkingBox-Bench task was included in the training pool; the benchmark was reserved for evaluation and is treated as out-of-distribution, though the shared framework and simulated-user machinery limit the transfer claim.

The repository release includes data preparation, rollout, reward, and Verl integration. The README points to docs for data preparation, the validated Qwen3.8 24-H100 configuration, and evaluation. The validated environment targets Linux and WSL with Python 3.12, Verl 0.9.0, and pinned commits for the ThinkingBox repositories. After installation, users prepare training data with data/prepare_data.py, launch Ray on three nodes with 8 H100s each, and run the training script. Evaluation requires merging the checkpoint, serving it through vLLM with the qwen3 reasoning and tool-call parsers, starting the OpenEnv compatibility proxy, and running thinkingbox-eval for the full 507 tasks x 20 attempts. The public sample task list is not the private 187-task pool used for the reported run.

The post closes with practical advice: probe before training, check parser parity between training and evaluation, and report both coverage and repeatability. It points to the ThinkingBox paper, the environment, the benchmark, and the training repository for further details.