Behind the scenes, two months later: How we ship Google Agent Skills like products

In my previous post, I took you behind the scenes of Google Agent Skills, covering how the project started, how we maintain quality at scale, and how we handle governance. In this post, I'll share what we've built since then: the automated quality gates that every change must pass, the AI review agent that inspects skill quality, and the lessons that we learned along the way. You can adopt similar gates and review rules in your own skill repository. Since my last post, the Google Agent Skills repository has passed 21,000 GitHub stars, up from 15,000. It now hosts over 170 skills from teams across Google, and we distribute them across multiple AI platforms. Building agent skills for a single developer or a small project is straightforward. You write a SKILL.md file, define a few instructions, test it manually, and it works. At our scale, manual workflows stop being effective. Unvetted edits can introduce subtle bugs, and it becomes difficult to catch issues before they reach users. The core philosophy: Ship skills like products One key lesson from my previous post: a skill is a living product, not a one-off document. In practice, that means we ship every skill update like a product release. It's tempting to treat a skill as documentation. But documentation explains concepts to a human reader, while a skill steers an AI agent's reasoning loop, tool selection, and execution path. A misplaced phrase in a skill can cause an agent to invoke invalid command flags, enter an infinite loop, or waste tokens. That makes every instruction edit a behavioral change. You wouldn't ship a product update without testing it, and the same goes for a skill. This principle drives three core requirements: Strict semantic versioning: Every behavioral change requires an explicit version bump. Deterministic automated validation: Linters and validators check every proposed change before a human or AI reviewer sees it. Mandatory evaluation evidence: Every change includes an evaluation report. It compares the skill against its previous version, or—if the skill is new—against an agent without the skill. Automated conformance gates: The pull request pipeline In my previous post, I briefly listed the checks that every skill passes on check-in: linters, link checkers, and AI-assisted checklists. Here's what's inside them, plus two gates that I haven't covered before: semantic versioning and evaluation suite validation. When an author submits a change to our internal skills repository, an automated continuous integration and continuous delivery (CI/CD) pipeline triggers immediately. Before a skill is exported to GitHub, it must pass these six automated gates and receive human approval: Structural conformance: Directory layout, allowed files, and taxonomy. Skill linters: Frontmatter, folder naming, and Markdown rules. Semantic versioning: Version bump for every behavioral change. Evaluation suite: Valid eval suite and a fresh eval report. Link validation: Links are valid. AI review agent: Skill quality and design standards. Human approval: Final approval from a human reviewer (a subject matter expert). Structural conformance and skill linters Our automated validation suite inspects every pull request for structural and syntactical correctness. Some of the rules include these requirements: Folder name and metadata name must match. SKILL.md and description length limits are enforced. All required fields in the frontmatter metadata are present. Only a curated list of files and directories is allowed within a skill directory. All Markdown files follow formatting rules. The main README.md file and the machine-readable index.json registry are up to date. Semantic versioning Every skill declares a semantic version in its SKILL.md frontmatter metadata: code_block <ListValue: [StructValue([('code', '---\r\nname: spanner-basics\r\nmetadata:\r\n version: "1.1.1"\r\n category: Databases\r\ndescription: ...\r\n---'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x7eff3579ee50>)])]> First, our automated checks verify that the version is a quoted string. YAML parses unquoted values like 2.0 as numbers, which causes unexpected parsing errors in tools that expect a string. Second, the validator checks the version transition itself. New skills must start at 1.0.0, and any change to an existing skill's instructions, scripts, or references requires a version bump or the pull request is blocked. Standard reset rules apply: bumping MAJOR (for example, 1.4.2 to 2.0.0) resets MINOR and PATCH to 0, and bumping MINOR (for example, 1.4.2 to 1.5.0) resets PATCH to 0. Why enforce strict versioning if the agent only reads the Markdown instructions? The version number is for the developer and their package tooling, not for the model at runtime. Skills don't auto-update in the background, which means there's no network check or latency overhead each time an agent invokes a skill. Instead, explicit versions let developers c

Excerpt shown under the publisher's RSS terms · 200 words · full text and figures are on the publisher's site