The question this post addresses is not whether agentic memory works, but how much of it a given model can actually use. The authors’ conclusion is that memory is a dose to calibrate, not a feature to switch on. Across eight models ranging from a 30B dense model to frontier proprietary systems, three recurring patterns emerged. Strong models with headroom want the full guideline set: DeepSeek-V3.2 (671B MoE) gained +9.5 percentage points in task completion when given all its self-mined guidelines. Smaller or weaker models get overwhelmed by a large set and do best with a compact high-confidence core plus a handful of task-relevant guidelines retrieved per task: gpt-oss-120b (117B MoE) gained +16.1pp with this selective approach, while the full set gained less and cost roughly 50% more tokens. Already-saturated models show no measurable gain; GLM-5 (745B MoE) sat at 87.5% task completion with or without memory. The article emphasizes that what places a model into one pattern is not simply parameter count—benchmark headroom, context-window size, architecture, guideline quality, and task distribution all appear to play a role, and separating those factors is ongoing work.

The memory mechanism itself is deliberately external. ‘Memory’ means a guideline set—strategies that worked, mistakes to avoid, edge cases—distilled from the agent’s own prior trajectories. ALTK-Evolve runs the loop: the agent attempts tasks, produces trajectories, ALTK-Evolve extracts behavioral guidelines from both successful and unsuccessful runs, consolidates them into a reusable set, and at inference time injects either the full set or a task-relevant selection. No model weights are updated, which makes the approach cheap to adopt and portable across all eight models tested.

The evaluation uses AppWorld, 585 multi-step tasks across nine simulated apps, with two metrics: TGC (Task Goal Completion), the share of individual tasks fully completed, and SGC (Scenario Goal Completion), a stricter all-or-nothing metric that counts a scenario as passing only if every variant of the task succeeds. SGC often moves more than TGC: DeepSeek’s SGC jumped +16.1pp against a +9.5pp TGC gain, because good guidelines particularly help clear every variant of a scenario. Even GPT-5.5 and Claude Opus 4.6, both near the ceiling on TGC, gained +7.2 and +7.1pp SGC respectively.

The three configurations are defined by what enters the context: baseline (no memory), full guideline set (every mined guideline injected on every ReAct step), and curated retrieval (a fixed high-confidence core plus task-relevant guidelines retrieved per task). All memory configurations draw from the same guideline set mined once from AppWorld‘s training split only; no test-split data is used. Because the number of guidelines a model mines depends on its capability, the article reports by strategy rather than raw counts.

Cost is a major practical concern, since a full set is re-sent on every step. Measurements show DeepSeek’s token use per task rose from 148K to 263K (+78%), and gpt-oss-120b‘s from 110K to 166K (+51%). Curated retrieval kept gpt-oss-120b at 116K tokens per task, just +5% over baseline, while delivering the largest accuracy gain. The article also notes that prompt caching can substantially cut effective cost in production, as the static portion of the guideline set is identical across steps and can be cached; keeping the shared prefix stable is worth engineering for. The number of ReAct steps stays roughly the same with and without memory (DeepSeek ≈18–19), so the added cost is input-token inflation, not longer trajectories.

The conclusion is that memory should be calibrated, not merely accumulated: weak models get a compact core plus a few task-specific lessons—which is also the cheapest option; strong models get the full guideline set, kept affordable via prompt caching; saturated models get no extra context until their remaining failure modes are better understood. Next steps include a learned selector trained on outcome signal to replace cosine similarity retrieval, teacher-distilled memory for models below a minimum capability baseline, evaluation beyond AppWorld, and controlled experiments to isolate context-window size from raw capability.