aiqbench analysis
This analysis highlights a narrowing performance gap between open-weight models like GLM-5.2 and DeepSeek-V4-Pro against proprietary frontier models in specific cyber tasks, though long-horizon chaining capabilities still favor closed systems. The UK government's AI Security Institute plans to evaluate Kimi K3 using this same metric once weights are released, signaling a shift toward assessing open-source viability for national security defense.
Import AI 465: Open vs closed gaps; Kimi K3; Demis’ big policy plan

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. Subscribe now UK government: Gap between open and closed weight models on cyber is shrinking:…The cyber-eschaton cometh…The UK government’s AI Security Institute (AISI) has analyzed the delta in […]
Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. UK government: Gap between open and closed weight models on cyber is shrinking:…The cyber-eschaton cometh…The UK government’s AI Security Institute (AISI) has analyzed the delta in cybersecurity capabilities between powerful proprietary models and open weight models. The results show that this year, the gap has shrunk. “This is our first public analysis of how far leading open weight models trail the closed cyber frontier,” AISI writes. “Recent open models GLM-5.2 and DeepSeek V4-Pro perform similarly to frontier closed models released 4 to 7 months before them – a narrower gap than the 6 to 10 months we measured through most of 2025.” Specific details: On a set of 70 evals for specific, narrow cyber capabilities, GLM-5.2 is closest to Claude Opus 4.6, which was released 4.3 months earlier, while DeepSeek-V4-Pro sits somewhere between Claude Opus 4.5 and GPT-5 (released in November and August 2025, respectively). “AISI intends to test Kimi K3 on this same basis, once its weights are publicly released,” AISI writes. The gap lengthens a bit for long-horizon cyber ranges, which…
Excerpt shown under the publisher's RSS terms · 200 words · full text and figures are on the publisher's site
Related coverage

[AINews] not much happened today
Latent Space · Oct 3, 2026

AISN #77: New Model Releases From OpenAI, SpaceXAI, and Meta
Center for AI Safety · Jul 21, 2026
Links #5: 2026/07
LessWrong · Jul 31, 2026
Open models recap: more on Kimi K3, Qwen 3.8, Xi's WAIC speech, distillation, the open-closed gap, and what's next
Interconnects · Jul 22, 2026