Beam by Reflection is now available on Hugging Face under Apache 2.0. It is the first model from the new US lab Reflection: a text-only mixture-of-experts model with 501B total parameters, 23B active per token, and a 1M-token context window, aimed at coding and agentic workloads. The release includes BF16, FP8, and NVFP4 checkpoints, day-0 support in transformers, vLLM, and SGLang, a Rust-based custom serving harness, and a speculative-decoding drafter model using DFlash2.

Architecturally, Beam has 52 decoder layers with hidden size 5,120. Forty-three layers use sliding-window attention over 512 tokens; nine global layers (every sixth and the last) attend to the full context. Sliding-window layers use RoPE, while global layers avoid positional encoding and apply scalable-softmax query scaling. Attention uses grouped-query attention with 48 query and 8 key-value heads. The first three layers are dense SwiGLU; the other 49 use an MoE block with 256 experts, each about 39M parameters, following DeepSeek-V3-style group-limited routing: all experts are scored, split into 8 groups, the 4 best groups are kept, and 8 experts are routed per token. There is no shared expert. The residual stream is also distinctive: each sublayer output is added with weight 1/sqrt(k), so the hidden-state scale stays constant through all 104 additions, and final logits pass through a tanh softcap at ±30.

Because only 8 experts run per token, compute cost is around 23B active parameters, but the full 501B remain in memory. The BF16 checkpoint is about 1 TB; NVFP4 shrinks it to roughly 280 GB and is designed for Blackwell GPUs, while FP8 targets Hopper and Ampere. Hybrid attention lowers KV cache growth from 208 KiB per token in an all-global design to about 36 KiB per token; at full 1M context that is roughly 36 GiB per sequence versus 208 GiB. Reflection-provided benchmarks show quality stays close across precisions; for example, SWE-Bench Verified is 81.6% in BF16, 80.2% in FP8, and 80.7% in NVFP4, while AIME 2026 sits around 97.7-98.0%.

The post shows several deployment paths. The transformers pipeline works with AutoModelForCausalLM; Beam‘s tokenizer carries a response template, so parse_response splits reasoning, final answer, and tool calls without ad-hoc regex. Multi-GPU loading uses expert parallelism with a DistributedConfig, placing each expert on one device while attention, dense layers, embeddings, and the LM head are replicated. vLLM serves the BF16 checkpoint with tensor parallelism 16, optionally with DFlash2 speculative decoding. SGLang recipes cover NVFP4 on 4 B300s with the trtllm_mha backend and FP8 on 8 H200s with fa3, both enabling DFlash2. Beam is also available through Hugging Face Inference Providers on Baseten, Fireworks, and Together.

For post-training, Hugging Face shares recipes in TRL and Miles. An SFT example uses LoRA adapters on Multilingual-Thinking; after one epoch, 16/16 held-out prompts reasoned in the requested language versus 3/16 for the base model. The Miles GRPO recipe, built with RadixArk, trains in BF16 with Megatron and SGLang rollouts; a full-parameter run on DAPO-Math-17K uses twelve 8xH200 nodes, streams optimizer state through NVMe, and evaluates on AIME25 every five rollouts. A LoRA GRPO variant fits on four 8xH200 nodes.

Finally, the announcement includes a $10,000 Hub credit contest and three demonstrations: Beam built a native macOS dictation app called BeamSpeak using Parakeet/MLX inside the Pi coding harness; a multi-agent scale experiment on ProgramBench compared one agent, five peers, and lead/subagent delegation; and a triage decision model was fine-tuned from Qwen3.5-0.8B after Beam‘s quota ran out, reaching 76.25% full correctness versus 4.69% zero-shot with an 11.9x latency reduction. The authors note the benchmarks were provided by the Reflection team, and the vibe evals are explicitly not benchmarks; low reasoning effort occasionally failed format or justification requirements.