Inkling 2 Mini: Looped Transformer MoE for Local Setups

Inkling 2 Mini is a new multimodal mixture-of-experts model from Thinking Machines Lab aimed at local setups. It has roughly 26B physical parameters with about 3B active per token, supports a 1M-token context, and accepts both image and text inputs. The release uses a mixed-precision format with NVFP4 and MXFP8 expert weights, and ships with a DSpark speculative-decoding drafter for additional speed-up at a small memory cost. The authors say it is much smaller than Inkling 2 Small and, in their vibe tests, at least comparable with Qwen 3.8 27B, while noting more validation is needed.

Architecturally, Inkling 2 Mini is a looped transformer: 24 physical decoder layers behave like 48 logical layers because the 12 middle layers run three times. Layers 0–5 run once, layers 6–17 run three times with the same weights, and layers 18–23 run once. The hidden size is 2,304 rather than Small’s 4,608, and each expert has intermediate width 1,152. Each layer visit routes a token to 8 experts selected from 128 per layer, while Small selects from 384; the router normalizes selected scores and scales the weights by 8. Each six-layer block has five sliding-window attention layers with a 1,024-token window and one full-attention layer; attention uses 24 query heads and 8 KV heads, with learned relative position logits, a learned sink, and an output gate. Every pass through the looped core starts from the same inputs, and information moves between passes through cross-pass messages. A small gating controller looks at the layer’s current input and the message, deciding how much input to forget and how much message to add, so each pass refines the previous one. During passes 2 and 3, each core layer attends over two residual blocks from the previous pass, with four rank-64 gates mixing messages into the query, key, value, and MoE inputs. Reusing core weights cuts the checkpoint size, but all 48 layer visits keep their own KV and short-convolution state. Looping has been explored before in architectures such as Nanbeige, but this cross-pass message design is the interesting mechanism.

Alongside the fixed-loop flagship, the team released a research artifact with the same architecture trained to accept a variable number of passes. Setting 1 or 2 passes instead of 3 makes inference considerably faster, with potentially slightly lower generation quality; the variant works but has not been evaluated in depth.

On benchmarks, Inkling 2 Mini is state-of-the-art in its class on the Artificial Analysis Intelligence Index v5 (AAv5). Compared with Gemma 4 31B, Muse Glimmer, Qwen3.8-27B, and Nemotron 3.5 Super VL, it leads in 32 of 48 benchmarks across coding and autoresearch, agentic search and tool use, vision interpretation and creation, reasoning and knowledge, and safety and alignment. For coding, it leads FrontierCode v2.0, ties Qwen3.8-27B on Terminal-Bench 4.0, and trails Qwen on AutomationBench-AA v1.0.6.

Inference support spans transformers, llama.cpp/Llama App, MLX, and SGLang. Transformers users can load the model with AutoModelForMultimodalLM; the research checkpoint exposes a text_config argument to set loop_num_passes. The processor supports image prompts, tool calling, and a reasoning_effort parameter with levels from none (0.0) through max (0.99), defaulting to high (0.9). Llama App provides GGUF IQ4_NL and Q8_0 quants for both models, and Linux/CLI users can serve them with llama serve. SGLang offers day-zero support and requires running on SM103; it can enable multimodality, and can use the three-layer DSpark draft with the DSPARK speculative-algorithm flag. The post’s SGLang benchmark script measures generation on a single NVIDIA B300 GPU with 1,024-token prompts and outputs at concurrency 1, 4, 16, and 64. MLX support has been upstreamed to mlx-lm, including multimodal image/text inputs and shared quants, and MLX can run the original checkpoint directly in its native NVFP4+MXFP8 scheme.

The post also demonstrates the model in agentic and creative tasks: a WebGL fragment-shader sunset flythrough from the same prompt for Inkling and Qwen, a procedural animated recreation of Van Gogh’s The Starry Night refined over 24 screenshot rounds, and a hand-drawn Trip Split wireframe turned into a working HTML expense app that passed all 13 Playwright checks. A 4-bit GGUF quant autonomously built a plant therapy SaaS website with SVGs and matching logic from a short prompt, and the model was paired with the Pi coding agent to build itself into a browser agent using Helium, Chromium, and Hugging Face Sandboxes, with traces and final file shared.

Welcome Inkling 2 Mini, Cutting-edge Performance for your Local Setup

View Original