ALTK-Evolve vs ACE: Same Lessons, Far Fewer Tokens on Agent Tasks

Both ACE (Agentic Context Engineering) and ALTK-Evolve improve LLM agent performance on multi-step tasks by mining past trajectories for reusable lessons—no weight updates, no human labels.

They agree on a core principle: never compress an agent’s hard-won experience into a short summary.

Instead, ACE keeps an itemized playbook with per-bullet helpful/harmful counters and injects it on every step regardless of model or task.

ALTK-Evolve consolidates near-duplicate lessons into typed guidelines (strategy, recovery, optimization) with provenance and support counts, then calibrates delivery per task and model—a small fixed core of high-support guidelines plus a handful selected for the current task.

This delivery difference drives token cost. On AppWorld with the same ReAct agent, tested in-house on two base models: for DeepSeek-V3.2, ACE achieves 80.4 TGC (634K tokens/task) vs ALTK-Evolve 89.

3 TGC (263K tokens); for gpt-oss-120b, ACE 54.8 TGC (777K tokens) vs ALTK-Evolve 56.0 TGC (116K tokens).

ALTK-Evolve matches or exceeds ACE’s accuracy at a fraction of the inference cost—about 40% of ACE’s tokens on the strong model and about one-seventh on the weaker one.

The by-difficulty breakdown shows that on harder tasks, where the agent must pick the right lesson rather than sift through all of them, curated retrieval pulls ahead.

The authors note that ACE’s own efficiency story is about building context cheaply, while ALTK-Evolve‘s is about serving it.

Results are single-run (pass@1) on AppWorld test_normal (168 tasks), and the ACE numbers are the authors’ own runs to keep the comparison controlled for model and harness; the two no-memory baselines differ (79.

8 vs 72.0 TGC) due to prompt-template differences, but the claims rest on the post-memory comparison and token savings.

The ALTK-Evolve library includes the full extraction, consolidation, and retrieval pipeline; a technical report provides complete method and ablations.

Thinking of ACE? We Can Do It with Fewer Tokens

View Original