agentsop-observability-setup
From the author
Enhancement-overlay skill — the DECISION + WIRING layer for LM observability that the single-backend skills [[langsmith]], [[phoenix]], [[mlflow]] do NOT cover. Each of those installs one backend; none of them help you DECIDE which backend fits your stack/scale/budget, nor give you a one-line autolog that turns it on fast. Use when starting any LM project, before the first deploy, or the moment someone asks "why did it do that?" and there are no traces to answer with. The skill picks a backend by stack (LangSmith for LangChain/LangGraph; Phoenix for OSS/local OpenTelemetry; MLflow for ML-shops already on MLflow; Langfuse for self-host), wires one-line autolog, verifies traces land, and adds eval hooks — instrumenting BEFORE you need it. Cross-links the first-debug-move skill [[agentsop-prompt-history-inspect]]. Do NOT activate to re-teach a backend you already chose (defer to its own skill), or for non-LM ML experiment tracking with no LLM calls (that is plain MLflow).
Install
npx skills add agentsope/SkillAlchemy --skill agentsop-observability-setup -a claude-code
Installs to .claude/skills/agentsop-observability-setup/ · Each command fetches the skill straight from GitHub with the open skills CLI.
Byte-identical to the scanned commit, with the repository’s licence and a SOURCE.txt.
Want your agent to suggest skills while you type? See how to use skills from your IDE.
Safety checks on SKILL.md
No warnings.
Why 86
aiqbench computes the trust score from public signals. It is not a security audit, so read a skill before you install it.
Similar skills
Closest to this one by what they do and when they are used.
- agentsop-tool-scopingagentsope/SkillAlchemyEnhancement overlay for multi-agent / tool-using coder agents. Encodes the per-agent tool- scoping discipline that role-based frameworks (CrewAI, LangChain) document only as a passing best-practice: which agent gets which tool, and why blanket-sharing every tool to every agent is a correctness and blast-radius risk. Activates when an agent system has tools AND there is more than one agent (or one agent holding many tools). Treat a tool as a capability grant; scope by least-privilege. ENHANCE overlay — read alongside [[crewai]], [[agentsop-http-tool-wrapping]], [[agentsop-llm-tool-idempotency]]. Search keywords: which tools per agent, least-privilege agent, agent tool access, tool permissions, limit agent tools, scope tools to roles.overlay_type: enhancementOther★ 445
- agentsop-streaming-outputagentsope/SkillAlchemyEnhancement-overlay decision protocol for STREAMING the output of long-running LLM / agent runs from the *backend*, not just wiring a typing animation in the UI. Activates when a coder agent must stream final tokens to a chat client, surface intermediate agent steps (which tool, which node, partial reasoning), emit custom tool-progress events, choose a transport (SSE vs WebSocket), or decide what to do when the client disconnects mid-stream. The langchain / langgraph skills mention stream modes but stop at "you can stream"; this skill encodes *what to stream, over what transport, and how to fail safely*.Design★ 445
- agentsop-llm-artifact-versioningagentsope/SkillAlchemyEnhancement overlay — version the WHOLE deployable LLM-app artifact as one bundle: prompts + compiled programs + model snapshot pins + retrieval config + eval-set version, versioned together so a deploy is reproducible and rollback is atomic. Activate when preparing to deploy an LLM app, when asking "what exactly is running in prod right now?", when a deploy must be reproducible months later, or when an incident needs a clean rollback. The core reframe: an LLM app artifact is NOT an ML model — it is a manifest over many independently-mutable parts, not one weights file. Do NOT activate for one-off prompt edits with no deploy, for a single-component demo, or where a vendor owns the whole prompt lifecycle. For versioning ONE compiled prompt use [[agentsop-per-model-artifacts]]; for the CI comparison mechanism use [[agentsop-regression-gate]]. Search keywords: prompt versioning, reproducible deploy, what is running in prod, rollback LLM app, model pinning, prompt registry, version prompts and config.DevOps★ 445
- agentsop-prompt-history-inspectagentsope/SkillAlchemyTool skill — the *first move* in any LM-debugging session: dump the actual rendered prompt the framework sent to the model, before changing anything else. Activate when an LM call produced an unexpected output (wrong answer, schema violation, refusal, truncation, cost spike, latency spike, infinite loop, "model got dumber after upgrade"). The skill enforces a 30-second inspect step BEFORE any prompt edit, model swap, retry, or temperature tweak. Cross-framework cheat sheet: DSPy `inspect_history`, LangGraph `get_state_history`, CrewAI `step_callback`, LangChain `set_debug`/`set_verbose`, Aider `/diff`+`--verbose`, raw OpenAI/Anthropic via `OPENAI_LOG=debug`/`ANTHROPIC_LOG=debug` or HTTPX event hooks. Do NOT activate for first-time prompt authoring, exploratory prompt design, or non-LM bugs.Design★ 445
- langsmith-observabilityOrchestra-Research/AI-Research-SKILLsLLM observability platform for tracing, evaluation, and monitoring. Use when debugging LLM applications, evaluating model outputs against datasets, monitoring production systems, or building systematic testing pipelines for AI applications.DevOps★ 13.9K
- agentsop-langgraphagentsope/SkillAlchemyDecision protocol for building, debugging, and operating LangGraph-based agent systems. Activates when a coder agent is asked to design a stateful LLM workflow, add human-in-the-loop, choose a multi-agent pattern (supervisor / swarm / hierarchical), pick a checkpoint backend, or migrate a fragile chain into a durable graph. LangGraph is positioned by its maintainers as a "low-level orchestration framework for building, managing, and deploying long-running, stateful agents" — this skill encodes the *when* and *why*, not the API.DevOps★ 445
Ratings & reviews
—
no reviews
Tried this skill? Rate it to help others. Sign in
Discussion
Join the discussion. Sign in