The article addresses a real gap: until now, no large open model has natively handled text, image, and audio together at scale with a 1M context window and agentic capabilities. Inkling by Thinking Machines aims to be that model—a ~1 trillion parameter multimodal MoE that understands all three modalities natively, without separate encoders for each, and targets reasoning across them. The tension is that building such a model is infrastructure-heavy, but the authors argue the open release makes multimodal reasoning accessible to a wide audience for fine-tuning and deployment.

Inkling is a decoder-only Mixture-of-Experts model with 975B total parameters and 41B active, using 256 experts and a shared expert sink. Its architecture is distinctive: hybrid attention alternates between sliding window and global attention in a 5:1 ratio, a short 1D convolution (SConv) processes local context before the transformer layers, and relative attention replaces RoPE. Multimodal input is handled by simple hierarchical MLP tower for images and discretized mel spectrograms for audio. The model supports Multi-Token Prediction (MTP) drafters for speculative decoding, speeding generation without accuracy loss. Day-0 inference support exists in transformers, SGLang, vLLM, and llama.cpp, with quantized NVFP4 and GGUF versions reducing VRAM drastically. Benchmark results place it competitively with top proprietary models on reasoning, coding, and agentic tasks like SWE-bench and MCP-Atlas.

For a serious builder, Inkling represents a pragmatic, open foundation for multimodal applications. Its simple but effective multimodal towers and hybrid attention offer a reproducible blueprint, while the 1M context and agentic capabilities (via Pi and OpenEnv) enable document processing and coding agents out of the box. The availability of quantized variants for local deployment and SLURM scripts for cluster inference lower the barrier to serious use. The architecture choices—especially the short convolution and relative attention—are worth studying for anyone building efficient long-context multimodal models.