aiqbench analysis
This study reveals critical vulnerabilities in the rapidly expanding OpenClaw agent skill ecosystem, where high download counts and automated security scanners fail to reliably identify malicious code or privilege escalation risks. The research demonstrates that governing these registries requires robust, transparent validation beyond simple metadata, as most listed skills lack human scrutiny despite their potential for executing dangerous system actions.
After the Party: Growth, Governance, and Security Scanning in the OpenClaw Agent Skill Ecosystem
arXiv:2609.17274v2 Announce Type: replace-cross Abstract: AI agents increasingly act through agent skills, i.e., natural-language instructions, that direct a host agent toward shell, network, credential, file, and process actions, and public registries distribute them at scale. In the first half of 2026, the OpenClaw AI agent went viral, and its public skill registry boomed: the observable stock nearly doubled in 91 days, and a majority of the listings visible in June were created in just two months. By the end of our study window, the wave had crested, and monthly listing creation and core-repository activity were falling from their spring peaks. This paper measures what the boom left behind, drawing on the OpenClaw Git history, its GitHub issues and pull requests, and three ClawHub registry snapshots. Attention is concentrated: the top 10% of skills received 46.93% of all downloads. No simple skill features (like size or download counts) remained a stable predictor of continued listing once creation cohort and skill age were controlled. Human scrutiny did not stay: 77.86% have zero stars and zero comments, while 85.06% of the readable skills carry privilege evidence. And automated cleanup is not ready: the three security scanners disagreed on 23,702 of the 61,990 skills they all cover. After human adjudication, weighted scanner sensitivity against the reference standard ranged from 21.67% to 61.06%. Governing fast-growing agent-skill registries cannot rely on simple metadata or single scanner scores; it requires robust, transparent measurement and independent validation.
Abstract:AI agents increasingly act through agent skills, i.e., natural-language instructions, that direct a host agent toward shell, network, credential, file, and process actions, and public registries distribute them at scale. In the first half of 2026, the OpenClaw AI agent went viral, and its public skill registry boomed: the observable stock nearly doubled in 91 days, and a majority of the listings visible in June were created in just two months. By the end of our study window, the wave had crested, and monthly listing creation and core-repository activity were falling from their spring peaks. This paper measures what the boom left behind, drawing on the OpenClaw Git history, its GitHub issues and pull requests, and three ClawHub registry snapshots. Attention is concentrated: the top 10% of skills received 46.93% of all downloads. No simple skill features (like size or download counts) remained a stable predictor of continued listing once creation cohort and skill age were controlled. Human scrutiny did not stay: 77.86% have zero stars and zero comments, while 85.06% of the readable skills carry privilege evidence. And automated cleanup is not ready: the three security scanners disagreed on 23,702 of the 61,990 skills they all cover. After human adjudication, weighted scanner sensitivity against the reference standard ranged from 21.67% to 61.06%. Governing fast-growing agent-skill registries cannot rely on simple metadata or single scanner scores; it requires robust, transparent measurement and independent validation.
To appear in IEEE Digital Library as the 33rd Asia-Pacific Software Engineering Conference (APSEC 2026) conference proceedings. Accepted version, not camera ready version
Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
arXiv:2609.17274 [cs.SE]
(or arXiv:2609.17274v2 [cs.SE] for this version)
Full text under the publisher's open licence · 275 words
Related coverage

NVIDIA GTC Taipei at COMPUTEX: Live Updates on What’s Next in AI
NVIDIA · May 21, 2026

Cloud Forensics and the Digital Crime Scene: How Investigation, Prevention, and Resilience Have Evolved
AppDirect · May 21, 2026
Links #3: 2026/06 Part 1
LessWrong · Jun 15, 2026

Netskope (NTSK) Q2 2027 Earnings Call Transcript
fool.com · Sep 10, 2026