ℹ️ Quick Answer: AI distillation attacks let competitors copy an AI model’s capabilities by mass-harvesting its outputs. Anthropic just caught three Chinese labs, DeepSeek, Moonshot AI, and MiniMax, running 16 million exchanges through 24,000 fake accounts to steal Claude’s reasoning, coding, and agent capabilities.
What’s Inside
- What Actually Happened
- How Three Labs Pulled Off AI Distillation Attacks at Scale
- Why AI Distillation Attacks Matter for Regular People
- The Bigger Picture
- Common Questions About AI Distillation
Imagine you spent years building a product. Real time, real money, real sweat. Then you find out a competitor has been secretly feeding your work into their own system to clone your results. Not inspired by your product. Literally using it to build a knockoff.
I get why Anthropic is angry. If someone used my work to train a competing product, I would be too. This goes beyond AI. It is a basic IP issue that anyone who has ever built anything can understand.
That is exactly what Anthropic says happened to Claude today.
What Actually Happened
Anthropic caught three Chinese AI labs running industrial-scale AI distillation attacks against Claude. They published a detailed report naming names, sharing numbers, and calling for industry-wide action.
The numbers are wild. DeepSeek generated over 150,000 exchanges targeting Claude’s reasoning abilities. They even had Claude write out step-by-step internal reasoning behind completed responses, basically manufacturing chain-of-thought training data on demand. Anthropic also found DeepSeek using Claude to generate censorship-safe rewrites of politically sensitive questions about dissidents and party leaders.
Moonshot AI (the company behind Kimi Claw, which we covered recently) hit 3.4 million exchanges. They went after coding, computer vision, and agentic tool use across hundreds of fake accounts.
MiniMax was the biggest offender at 13 million exchanges focused on agentic coding and tool orchestration. Anthropic actually caught them mid-operation before they shipped the model they were training. When Anthropic released a new Claude model during the campaign, MiniMax pivoted within 24 hours and redirected half their traffic to capture the latest capabilities.
How Three Labs Pulled Off AI Distillation Attacks at Scale
AI distillation attacks at this scale required serious infrastructure. The three labs used commercial proxy services that resell access to Claude and other frontier models.
Think of it like a massive fake ID operation for AI. One proxy network ran over 20,000 fraudulent accounts simultaneously, mixing distillation traffic with legitimate customer requests to avoid detection. Anthropic calls these “hydra cluster” architectures. You ban one account, another pops up instantly. The labs distributed requests across Anthropic’s API and third-party cloud platforms, making it look like normal usage unless you knew what patterns to search for.
Anthropic traced the attacks through IP addresses, request metadata, shared payment methods, and corroboration from other AI companies seeing the same behavior. In DeepSeek’s case, they linked accounts to specific researchers at the lab.
Why AI Distillation Attacks Matter for Regular People
Here is the part that goes beyond corporate drama. AI distillation attacks strip out safety guardrails. Companies like Anthropic, OpenAI, and Google spend significant resources building systems that prevent their models from helping with things like bioweapons development or sophisticated cyberattacks. When a competitor distills those capabilities into their own model, those protections often vanish entirely.
Anthropic is also making a direct argument about export controls. The U.S. restricts advanced chip sales to China specifically to maintain an AI advantage. If Chinese labs can just copy American models through distillation, those restrictions look pointless. Anthropic’s argument is actually the opposite. These attacks prove chips matter even more because distillation at scale still requires serious compute to run millions of API calls and train on the results.
The Bigger Picture
Anthropic is not alone here. OpenAI made similar claims about Chinese developers just last week. The pattern is becoming hard to ignore. Remember when Chinese models started beating Claude on coding benchmarks and everyone was impressed? Stories like this add new context to those leaderboard results.
Anthropic says they are building better detection systems, sharing intelligence with other labs and authorities, and developing countermeasures that would make distilled outputs less useful for training. But they are also honest that no single company can solve this alone.
Whether you see this as corporate espionage, IP theft, or just the messy reality of a global AI race depends on your perspective. But the principle feels clear to me. If your product only exists because you secretly copied someone else’s work, that is a problem no matter what industry you are in.
Common Questions About AI Distillation
What is AI distillation?
AI distillation is when you train a smaller, less capable model using the outputs of a more powerful one. It is a legitimate technique that companies like Anthropic and OpenAI use to create cheaper versions of their own models. It becomes illicit when competitors use it to steal capabilities from rival models without permission.
Is AI distillation always illegal?
No. Distillation itself is a standard machine learning technique. It crosses the line when it violates terms of service, uses fraudulent accounts, or circumvents regional access restrictions. What Anthropic describes is the unauthorized, industrial-scale extraction of capabilities from a competitor’s product.
Why should regular people care about AI distillation attacks?
Because distilled models typically lose the safety guardrails built into the original. That means dangerous capabilities like helping with cyberattacks or bioweapons could end up in models with zero protections. If those models get open-sourced, anyone can access them.
Related reading: Microsoft Says AI Deception Is Getting Harder to Catch | The $3 Chinese Model That Beat Claude at Coding | New to AI? Start here
Leave a Reply