Experts Skeptical That Distillation Gave Kimi K3 Its Advanced Capabilities

White House science advisor Michael Kratsios accused Moonshot, the Chinese company behind the open-weight LLM Kimi K3, of copying Anthropic‘s Fable model through distillation while using export-restricted chips.

Kratsios wrote that such industrial distillation is unacceptable, but provided no evidence. Experts are skeptical that distillation alone explains Kimi K3‘s advanced capabilities.

Braden Hancock (Laude Institute, Snorkel AI co-founder) noted that Fable has only been publicly available since July 1st, making it unlikely that Moonshot could distill enough data, train, and release Kimi K3 in two weeks.

Nathan Lambert (AI2) argued that distillation is becoming less impactful as Chinese models approach the frontier and shift toward reinforcement learning.

He explained that distillation via supervised fine-tuning (SFT) mainly affects a model’s ‘manners,’ not core capabilities, and that replicating Fable-like abilities would require large-scale RL runs that are prohibitively expensive and time-constrained via APIs.

While Anthropic previously accused Moonshot (along with DeepSeek and MiniMax) of systematic distillation—detected via IP addresses and unusual query patterns—Hancock emphasized that Chinese teams possess deep technical expertise; Moonshot‘s co-founder is a CMU PhD.

The article also notes a parallel allegation that Moonshot obtained banned Nvidia Grace Blackwell 300 chips, possibly through a black market, with calls for stronger know-your-customer rules for data centers.

The Biden administration proposed such rules in 2024, but no progress has been made under Trump.

Experts say exploiting Anthropic's Fable isn't how Kimi K3 got so good | TechCrunch

View Original