agent-evals
From the author
Build an eval suite that can actually detect a regression — cases pulled from real traffic, graders that check properties rather than vibes, a recorded baseline, and per-case diffs in both directions. Use before claiming a prompt, model or agent change is an improvement, when agent behaviour must not regress, or when someone reports "it seems better" after eyeballing a handful of outputs. Not for tracing what one run did (llm-observability), not for testing deterministic code, and never as evidence that behaviour the suite does not measure is safe.
Install
npx skills add nahid-sparktales/agent-dispatcher --skill agent-evals -a claude-code
Installs to .claude/skills/agent-evals/ · Each command fetches the skill straight from GitHub with the open skills CLI.
Byte-identical to the scanned commit, with the repository’s licence and a SOURCE.txt.
Want your agent to suggest skills while you type? See how to use skills from your IDE.
Safety checks on SKILL.md
No warnings.
Why 82
aiqbench computes the trust score from public signals. It is not a security audit, so read a skill before you install it.
Similar skills
Closest to this one by what they do and when they are used.
- agent-evaluationthiientv/godmodeDesigns and runs reproducible evaluations for AI agents, prompts, tools, skills, and model-backed workflows using realistic datasets, isolated baselines, objective assertions, rubric grading, trajectory analysis, cost/latency tracking, and regression comparison. Use when measuring agent quality, optimizing skill triggering, comparing prompts or models, or gating an AI feature release. Not for ordinary deterministic unit tests.Testing★ 99
- evals-bootstrapundefined-ui/second-brain-osScaffold a first eval suite for an agent: mine real failures into cases, write behavioural checks over traces, and generate the runner. Use when the user wants evals, regression tests for an agent, a golden set, or asks how to know a prompt/model change did not break things. Do NOT use for a single task's done-check (goal-test) or for auditing context (context-audit).Testing★ 1.2K
- agent-harnessborghei/Claude-SkillsTest and evaluation harness for AI agents — scenario suites, deterministic replay, regression diffing, cost and latency budgets. Use when agent quality is vibe-checked, before shipping a prompt or model change, or when evals drift.Testing★ 1K
- llm-observabilitynahid-sparktales/agent-dispatcherSee what an agent actually did — one trace per run with nested model, tool and retrieval spans, token and latency accounted per step, and failures clustered by mechanism instead of read one at a time. Use when an agent misbehaves in ways you cannot reproduce, when cost or latency is unexplained, when "it sometimes fails" is the whole bug report, or before writing evals when you do not yet know which failures exist. Not for judging whether an output is correct (agent-evals), and not a replacement for the application's own monitoring.DevOps★ 59
- agent-evalericrisco/rsc-harnessUse when measuring whether an LLM or agent system actually got better and gating merges on it: golden sets, fixing an inflated LLM-as-judge, scoring RAG (faithfulness, contextual recall) or agent trajectories (tool correctness, completion), or picking an eval framework. NOT building the agent loop, tools or RAG plumbing (that is `building-agents`).Other★ 395
- testing-llm-featuresjaktestowac/awesome-copilot-for-testersBuilds an offline eval suite for LLM-powered features: golden reference cases per prompt template, deterministic scoring where possible, a regression gate on prompt, model and retrieval changes, and CI wiring with promptfoo, Vitest or deepeval. Use when a product ships prompts, agents or RAG, when a model or prompt upgrade needs a regression check, when LLM output is currently verified by someone eyeballing it, or when asked how to test a feature whose output is non-deterministic.Testing★ 118
Ratings & reviews
—
no reviews
Tried this skill? Rate it to help others. Sign in
Discussion
Join the discussion. Sign in