DeepSeek used distillation to train its R1 and V3 models

Since at least late 2024, companies including DeepSeek, Alibaba, and Moonshot AI have been systematically extracting data from U.S. frontier AI models like Claude, GPT, Gemini, and Grok. These China-based firms have reportedly processed billions of tokens across millions of requests using a technique called knowledge distillation, a practice the NSA, CISA, and FBI describe as forming “the core—not merely a supplement—of their AI development strategy.” DeepSeek, for example, has run organized distillation campaigns since 2024 specifically targeting reasoning capabilities and specialized optimizations to train its R1 and V3 models, while Alibaba used the technique to improve its Qwen family of AI models. This industrial-scale activity represents a systematic extraction of proprietary functionalities, posing a threat to U.S. technological leadership. DeepSeek R1 and V3 Training via U.S. Model Distillation DeepSeek ran an organized distillation campaign beginning in late 2024 to accelerate development of its R1 and V3 models, specifically targeting U.S. frontier AI models for synthetic training data. The company did not limit its efforts to broad data acquisition, but instead focused on extracting proprietary functionality and reasoning capabilities to minimize compute and research expenses. Between…