analiza aiqbench
SkillSieve presents a novel hierarchical framework for securing autonomous agents by combining lightweight regex filtering with deep LLM analysis and model jury debates. The research demonstrates high efficacy in detecting malicious skills across diverse modalities, including natural language instructions and executable code, while maintaining low operational costs on accessible hardware. This work addresses critical safety challenges inherent to the rapid expansion of agentic workflows where traditional security tools often fail due to modality limitations.
SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills
arXiv:2604.06550v3 Announce Type: replace-cross Abstract: Agent skills combine natural-language instructions with executable code while inheriting an agent's filesystem, credential, and network access. Attacks can span prose and files, whereas regex and code-only analyzers cover only one modality. SkillSieve applies three progressively deeper layers: recall-oriented regex, AST, and metadata triage; four parallel LLM security sub-tasks; and an independent three-model jury with debate on disagreement. We evaluate 49,592 real ClawHub skills, a 390-skill labeled benchmark, and 100 adversarial samples across five evasion techniques on a 440 USD ARM board. The full pipeline achieves F1 = 0.929 (precision 0.912, recall 0.945) at an average cost of $0.006 per skill. An optional XGBoost fast path reduces Layer-2/3 calls by 32% with a 1.7-point F1 decrease while preserving recall. On 52 Feishu/Lark packages, Layer 2 reclassifies 13 of 14 Layer-1 flags as safe after contextual analysis; we also deploy the system as a Feishu chat bot. Code, labels, and aggregate results are open-sourced.
Abstract:Agent skills combine natural-language instructions with executable code while inheriting an agent's filesystem, credential, and network access. Attacks can span prose and files, whereas regex and code-only analyzers cover only one modality. SkillSieve applies three progressively deeper layers: recall-oriented regex, AST, and metadata triage; four parallel LLM security sub-tasks; and an independent three-model jury with debate on disagreement. We evaluate 49,592 real ClawHub skills, a 390-skill labeled benchmark, and 100 adversarial samples across five evasion techniques on a 440 USD ARM board. The full pipeline achieves F1 = 0.929 (precision 0.912, recall 0.945) at an average cost of $0.006 per skill. An optional XGBoost fast path reduces Layer-2/3 calls by 32% with a 1.7-point F1 decrease while preserving recall. On 52 Feishu/Lark packages, Layer 2 reclassifies 13 of 14 Layer-1 flags as safe after contextual analysis; we also deploy the system as a Feishu chat bot. Code, labels, and aggregate results are open-sourced.
Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
arXiv:2604.06550 [cs.CR]
(or arXiv:2604.06550v3 [cs.CR] for this version)
Text integral sub licența deschisă a publicației · 168 cuvinte
Subiecte conexe
Beyond Maliciousness: Static and Runtime Attack-Surface Profiling of Agent Skills
arXiv · 15 oct. 2026
SKILLLITE: Evidence-Guided Malicious Skill Auditing with Compact LLMs
arXiv · 30 sept. 2026
Four ways AI has fundamentally changed the threat landscape in 2026
Sysdig Blog · 21 iul. 2026
GitLab 19.4 release notes
GitLab · 17 sept. 2026