agent-evals
De la autor
Build an eval suite that can actually detect a regression — cases pulled from real traffic, graders that check properties rather than vibes, a recorded baseline, and per-case diffs in both directions. Use before claiming a prompt, model or agent change is an improvement, when agent behaviour must not regress, or when someone reports "it seems better" after eyeballing a handful of outputs. Not for tracing what one run did (llm-observability), not for testing deterministic code, and never as evidence that behaviour the suite does not measure is safe.
Instalare
npx skills add nahid-sparktales/agent-dispatcher --skill agent-evals -a claude-code
Se instalează în .claude/skills/agent-evals/ · Fiecare comandă descarcă skill-ul direct de pe GitHub cu utilitarul open source skills.
Identic octet cu octet cu commit-ul scanat, cu licența depozitului și un fișier SOURCE.txt.
Vrei ca agentul să-ți propună skill-uri în timp ce scrii? Vezi cum folosești skill-urile din IDE.
Verificări de siguranță pe SKILL.md
Niciun avertisment.
De ce 82
aiqbench calculează scorul de încredere din semnale publice. Nu este un audit de securitate, așa că citește un skill înainte să-l instalezi.
Skill-uri similare
Cele mai apropiate de acesta prin ce fac și când sunt folosite.
- agent-evaluationthiientv/godmodeDesigns and runs reproducible evaluations for AI agents, prompts, tools, skills, and model-backed workflows using realistic datasets, isolated baselines, objective assertions, rubric grading, trajectory analysis, cost/latency tracking, and regression comparison. Use when measuring agent quality, optimizing skill triggering, comparing prompts or models, or gating an AI feature release. Not for ordinary deterministic unit tests.Testare★ 99
- evals-bootstrapundefined-ui/second-brain-osScaffold a first eval suite for an agent: mine real failures into cases, write behavioural checks over traces, and generate the runner. Use when the user wants evals, regression tests for an agent, a golden set, or asks how to know a prompt/model change did not break things. Do NOT use for a single task's done-check (goal-test) or for auditing context (context-audit).Testare★ 1.2K
- agent-harnessborghei/Claude-SkillsTest and evaluation harness for AI agents — scenario suites, deterministic replay, regression diffing, cost and latency budgets. Use when agent quality is vibe-checked, before shipping a prompt or model change, or when evals drift.Testare★ 1K
- llm-observabilitynahid-sparktales/agent-dispatcherSee what an agent actually did — one trace per run with nested model, tool and retrieval spans, token and latency accounted per step, and failures clustered by mechanism instead of read one at a time. Use when an agent misbehaves in ways you cannot reproduce, when cost or latency is unexplained, when "it sometimes fails" is the whole bug report, or before writing evals when you do not yet know which failures exist. Not for judging whether an output is correct (agent-evals), and not a replacement for the application's own monitoring.DevOps★ 59
- agent-evalericrisco/rsc-harnessUse when measuring whether an LLM or agent system actually got better and gating merges on it: golden sets, fixing an inflated LLM-as-judge, scoring RAG (faithfulness, contextual recall) or agent trajectories (tool correctness, completion), or picking an eval framework. NOT building the agent loop, tools or RAG plumbing (that is `building-agents`).Altele★ 395
- testing-llm-featuresjaktestowac/awesome-copilot-for-testersBuilds an offline eval suite for LLM-powered features: golden reference cases per prompt template, deterministic scoring where possible, a regression gate on prompt, model and retrieval changes, and CI wiring with promptfoo, Vitest or deepeval. Use when a product ships prompts, agents or RAG, when a model or prompt upgrade needs a regression check, when LLM output is currently verified by someone eyeballing it, or when asked how to test a feature whose output is non-deterministic.Testare★ 118
Note și recenzii
—
nicio recenzie
Ai încercat acest skill? Notează-l ca să-i ajuți pe alții. Autentifică-te
Discuție
Intră în discuție. Autentifică-te