nemo-evaluator-sdk
De la autor
Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.
Instalare
npx skills add Orchestra-Research/AI-Research-SKILLs --skill nemo-evaluator-sdk -a claude-code
Se instalează în .claude/skills/nemo-evaluator-sdk/ · Fiecare comandă descarcă skill-ul direct de pe GitHub cu utilitarul open source skills.
Identic octet cu octet cu commit-ul scanat, cu licența depozitului și un fișier SOURCE.txt.
Vrei ca agentul să-ți propună skill-uri în timp ce scrii? Vezi cum folosești skill-urile din IDE.
Verificări de siguranță pe SKILL.md
Niciun avertisment.
De ce 93
aiqbench calculează scorul de încredere din semnale publice. Nu este un audit de securitate, așa că citește un skill înainte să-l instalezi.
Skill-uri similare
Cele mai apropiate de acesta prin ce fac și când sunt folosite.
- evaluating-llms-harnessOrchestra-Research/AI-Research-SKILLsEvaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.Altele★ 13.8K
- nemo-guardrailsOrchestra-Research/AI-Research-SKILLsNVIDIA's runtime safety framework for LLM applications. Features jailbreak detection, input/output validation, fact-checking, hallucination detection, PII filtering, toxicity detection. Uses Colang 2.0 DSL for programmable rails. Production-ready, runs on T4 GPU.Altele★ 13.8K
- llm-evaluationwshobson/agentsImplement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.Testare★ 41.4K
- phoenix-observabilityOrchestra-Research/AI-Research-SKILLsOpen-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.DevOps★ 13.8K
- langsmith-observabilityOrchestra-Research/AI-Research-SKILLsLLM observability platform for tracing, evaluation, and monitoring. Use when debugging LLM applications, evaluating model outputs against datasets, monitoring production systems, or building systematic testing pipelines for AI applications.DevOps★ 13.8K
- tensorrt-llmOrchestra-Research/AI-Research-SKILLsOptimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.DevOps★ 13.8K
Note și recenzii
—
nicio recenzie
Ai încercat acest skill? Notează-l ca să-i ajuți pe alții. Autentifică-te
Discuție
Intră în discuție. Autentifică-te