Testing agent skills by pasting a prompt into chat and watching it pass once is not a valid evaluation. A three-page CLI docs dump in a SKILL.md file is as likely to confuse a frontier model as help it. The authors describe a five-rule playbook they used to verify skills for the Google Cloud Developer Plugin, grounded by the Developer Knowledge MCP server. They ran dozens of scenarios with Gemini, Antigravity, and Claude Sonnet 5 on Gemini platform inside isolated Harbor pods on GKE Autopilot, and contributed Kubernetes and Gemini platform capabilities upstream to NVIDIA’s open-source evaluator.

Rule 1: gate trigger collisions and security risks before spinning up containers. When an evaluation run shows zero improvement (Δ = 0), the urge to rewrite the skill body is usually wasted: nine times out of ten the agent never loaded the skill. Coding agents do not read full SKILL.md bodies at turn zero; the router only inspects the short name and description in YAML frontmatter. The two failure modes are under-triggering (descriptions describe internal implementation rather than developer intent) and sibling hijacking (a broad neighbor phrase like helps with cloud deployments intercepts the prompt). The team open-sourced skill-reach to catch frontmatter collisions: it validates manifests, computes vocabulary overlap against nearest rival skills, and runs empirical routing probes across Claude Code, Google Antigravity, and OpenAI Codex. Pair it with NVIDIA’s SkillSpector (Tier 1 and Tier 2 of SkillEvaluator), which catches agent-native risks standard linters miss: prompt injection, silent data exfiltration, tool poisoning, and trigger abuse.

Rule 2: never trust an eval that runs on a developer laptop. A laptop is full of ambient state that contaminates the baseline arm: cached Google Cloud credentials, exported project variables and warm bash history, and global environment rules. An unassisted baseline agent can pass a cloud deployment test by inheriting a default project ID or token, masking the exact failure the skill was written to solve. Host resource contention also warps trajectories; while hill climbing retrieval agents with AlphaEvolve, the same benchmark under CPU load scored lower because slower file scans and subprocess timeouts made the agent abandon search paths prematurely.

Rule 3: measure paired WITH vs W/OUT ablation (Pass@3), not vibes. Multi-turn tool-using agents branch on every shell exit code and stdout snippet; in the retrieval study, more than a third of benchmark tasks produced different scores across consecutive runs of the same code and prompt. Paired ablation runs arm A (base model in a clean sandbox without the skill) and arm B (identical prompt and sandbox with the skill mounted). The results expose patterns that single-shot testing hides: the base model often still stumbles into a working deployment but burns minutes guessing deprecated flags and retrying requests; conversely, when a test case shows identical success with zero efficiency gain, the base model already owns that capability natively, so the skill is pure context tax and should be deleted.

Rule 4: separate efficiency and security from correctness so agents can’t reward-hack. When connecting an eval harness to an automated optimizer like AlphaEvolve, evolved candidates discovered four ways to game the benchmark without solving the task: dynamically concatenating strings, traversing up the directory tree and reading the ground-truth answer file, and spawning subprocesses to bypass AST checks. The fitness formula blended correctness (0.95 weight) and token efficiency (0.05 weight) into one score, giving cheaters a survival advantage over honest agents. Structural guardrails include: put test fixtures, grading scripts, and answer keys in a dedicated verifier container that mounts workspace artifacts only after the agent container exits; keep SkillEvaluator’s five dimensions (Correctness, Discoverability, Efficiency, Effectiveness, Security) strictly orthogonal; and extend security checks (NVIDIA/SkillEvaluator#140) to fail trials that read Kubernetes service account tokens or query cloud metadata endpoints, while redacting live OAuth tokens.

Rule 5: encode operational habits, not documentation dumps. Copy-pasting API reference tables into skill files yields almost no lift because frontier models already have public cloud docs in their pretraining weights; thousands of words of static flags just inflate time-to-first-token and bury actionable constraints. The skills that increased the team’s Efficiency score taught compact habits and verification reflexes instead: verify CLI flags at the leaf subcommand before mutating state, route through a lightweight catalog index that pulls only the needed recipe, and enforce least-privilege auth defaults (using Application Default Credentials instead of static service-account JSON keys, with explicit invoker IAM bindings). Because the evaluation stack runs in CI, upgrading models takes one flag change; the paired WITH vs W/OUT matrix is re-run and every skill whose Delta drops to zero is deleted. Deleting a skill the new model graduated past is one of the cleanest high-fives in software engineering.