Highlights
Most general purpose LLMs are far too conservative
A licensed professional decides what correct is in clinical edge cases
Open sourced 200 input and 100 output guardrail scenarios clinically reviewed
Evals-Driven Development for a Mental Health AI Coach
SonderMind‘s Sonder is a clinically grounded AI coach for mental health support, built on a modular, eval-driven architecture.
The system uses separate input and output guardrails as LLM-as-a-judge calls, designed to be robust against prompt injection and easier to evaluate.
The critical innovation is a closed-loop clinical feedback process: licensed therapists annotate real conversation traces, which are automatically converted into typed evals that gate every release.
This ensures that a clinician’s judgment defines correctness, not heuristics.
The guardrails are calibrated to avoid over-triggering—they prioritize correct triggers over more triggers, distinguishing between active crisis, past trauma, and general support.
The agent harness includes memory, personalization, analytics, and alerting, with safety as the primary objective.
SonderMind has open-sourced 200 input and 100 output guardrail scenarios, clinically reviewed against real conversation patterns, to provide a shared baseline for the industry.
The design philosophy centers on the human: benchmarks are drawn from real failure modes, and the system avoids chasing perfection at the expense of user care.
SonderMind‘s Sonder is a clinically grounded AI coach for mental health support, built on a modular, eval-driven architecture.
The system uses separate input and output guardrails as LLM-as-a-judge calls, designed to be robust against prompt injection and easier to evaluate.
The critical innovation is a closed-loop clinical feedback process: licensed therapists annotate real conversation traces, which are automatically converted into typed evals that gate every release.
This ensures that a clinician’s judgment defines correctness, not heuristics.
The guardrails are calibrated to avoid over-triggering—they prioritize correct triggers over more triggers, distinguishing between active crisis, past trauma, and general support.
The agent harness includes memory, personalization, analytics, and alerting, with safety as the primary objective.
SonderMind has open-sourced 200 input and 100 output guardrail scenarios, clinically reviewed against real conversation patterns, to provide a shared baseline for the industry.
The design philosophy centers on the human: benchmarks are drawn from real failure modes, and the system avoids chasing perfection at the expense of user care.
