Agent skills are markdown instructions that steer coding agents, but hand-writing them makes it hard to know which instructions actually help. The authors set up a search loop that treats a skill as the program and uses real agent evaluations as the scoring function.

The loop combines AlphaEvolve and Harbor. AlphaEvolve proposes a rewritten version of a SKILL.md by operating on text placed between markers in a seed program; it runs in Google Cloud and never executes the user’s code. Harbor is the evaluator: it runs OpenCode with Gemini 3.8 Flash on tasks from Terminal-Bench 2.0, each trial in its own container on GKE with one pod per trial. A single search ran thousands of agent trials. For each candidate, the evaluator returns a score for passes and failures plus short notes on tasks that failed or timed out, and AlphaEvolve sends those notes back to Gemini to generate the next version.

They emphasize score trustworthiness because agent scores are noisy. In an early run, model latency explained much of the variance in candidate scores. The fix was to run each candidate and the baseline skill on the same tasks at the same time, so the score is the difference rather than an absolute value. They also held out unseen tasks for a final check. The best skill AlphaEvolve found read differently from the baseline checklist; it became a protocol built around the time limit.

The results were modest but clear: a 5% improvement after just a few iterations. The search focused on the most common failure — about a third of its trials hit the time limit, according to evaluator notes. They also found that the skill added value only when it supplied what the agent lacked: the baseline skill made no measurable difference, while a skill holding the answers added 33 points on the dev tasks. Their hypothesis is that today’s models and coding agents are already tuned for coding work, so general advice has little left to add, and the remaining room is in specifics that a search can find.

Running the search was harder than building the loop. After about 10 hours, no new candidate scored; slow model responses held back every evaluation and full agent logs filled the controller’s disk. The authors advise planning for slow model windows and log storage before long searches. They are also extending the approach to enterprise agent harnesses — system prompt, policy instructions, and tools — where the agent lacks business rules. In one small study on customer service test cases for a fictional bank, AlphaEvolve found and fixed a planted policy rule and carried the fix to unseen cases; other small studies were mixed, and final results are still pending.