design-ai-benchmarking
De la autor
Use when designing a study that benchmarks AI systems against a human-expert panel, before data collection. Plans the arms, decoupled rubrics with anchors, planted calibration probes, reviewer panel, inter-rater reliability targets and LLM-as-judge versus human adjudication.
Instalare
npx skills add Aperivue/medsci-skills --skill design-ai-benchmarking -a claude-code
Se instalează în .claude/skills/design-ai-benchmarking/ · Fiecare comandă descarcă skill-ul direct de pe GitHub cu utilitarul open source skills.
Identic octet cu octet cu commit-ul scanat, cu licența depozitului și un fișier SOURCE.txt.
Vrei ca agentul să-ți propună skill-uri în timp ce scrii? Vezi cum folosești skill-urile din IDE.
Verificări de siguranță pe SKILL.md
- Include scripturi pe care nu le-am verificatScăzut
De ce 80
aiqbench calculează scorul de încredere din semnale publice. Nu este un audit de securitate, așa că citește un skill înainte să-l instalezi.
Skill-uri similare
Cele mai apropiate de acesta prin ce fac și când sunt folosite.
- design-studyAperivue/medsci-skillsUse when checking a radiology or medical AI study design before drafting or submission. Reviews the analysis unit, cohort logic, leakage risks, comparator, validation strategy and reporting-guideline fit. AI-vs-expert benchmarks are /design-ai-benchmarking.Design★ 349
- llm-evaluation-designnaodeng/awesome-qa-skillsUse this skill when you need to design LLM evaluation datasets, judges, metrics, and human-review boundaries; triggers include LLM Eval 设计 and llm evaluation design.Design★ 279
- llm-evaluationwshobson/agentsImplement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.Testare★ 41.6K
- align-humanagentscope-ai/OpenJudgeUse when the user has a judge/grader and human-labeled data, and wants to measure how well the judge agrees with humans, detect systematic biases, determine whether automatic evaluation can replace human review, or build a human-reduction roadmap. Also use when the user mentions calibration, TPR/TNR, judge validation, inter-rater agreement, Cohen's kappa, bias detection, or "is my automatic evaluation trustworthy." Merges the calibrate and align functions into one skill.Planificare★ 888
- mltelagod/code-abyssMachine learning and LLM engineering judgment, distilled from a stronger model - invoke when DECIDING whether/how to use ML or an LLM for a task (prompt vs RAG vs fine-tune vs classical); working with training/eval data or labels; building or reviewing evals for models and LLM features; designing RAG, structured output, or agent pipelines; or diagnosing why a model/LLM feature underperforms. Method-selection ladder, data and leakage discipline, eval-as-spec rules, LLM-era craft, and a trap catalog.Planificare★ 242
- assessment-designcbrock84/headcountBuilds questions and assessments that measure what they claim — matching item type to the thing being assessed, writing distractors that diagnose, and telling a question that measures learning from one that measures reading speed or test-taking. Use this to write or review assessment items, design a unit check or diagnostic, work out why scores do not match classroom performance, or decide how much evidence a claim about mastery actually needs.Design★ 2.1K
Note și recenzii
—
nicio recenzie
Ai încercat acest skill? Notează-l ca să-i ajuți pe alții. Autentifică-te
Discuție
Intră în discuție. Autentifică-te