Muse Spark Open is Meta‘s open-weight, full-size entry in the Muse series, following the small dense Muse Glimmer. It is a mixture-of-experts model that accepts text, images, audio, and video and generates text. The model has approximately 816B total parameters with roughly 42B active per token, a context window of up to 1M tokens, and checkpoints in BF16 and NVFP4. Meta positions it as a coding- and agentic-focused model trained on real developer workflows such as multi-file refactors and long debugging sessions; it can read screenshots, listen to voice notes, and watch screen recordings while working on code.

The language model has 62 layers, a hidden dimension of 8192, and grouped-query attention with 128 query heads and 16 key-value heads, each of dimension 64. It applies QK-norm to queries and keys, and each block combines input and output through a learned residual gate. The dense feed-forward network is replaced by an MoE block with 256 experts per layer, 8 selected per token. Each expert projects 8192 to 4096 and back to 8192, then normalizes its own output. The experts hold roughly 97% of the total parameters; because only 8 experts run per token, the compute cost is that of a ~42B model, but deployment must keep all 816B parameters in memory. The BF16 checkpoint is about 1.5 TB in safetensors; the NVFP4 checkpoint stores parameters in 4-bit precision for NVIDIA Blackwell GPUs.

A 1.8B-parameter perception encoder handles images and video, splitting each image into 14×14 patches, using learned absolute embeddings and 2D RoPE, and alternating full attention every fourth layer with sparse attention elsewhere. Video uses the same encoder. The audio encoder uses a convolutional pyramid with patch sizes [16, 6, 5, 4] and emits one audio token every 80 ms, giving 12.5 tokens per second. Long context relies on grouped-query attention: each token adds 126,976 KV-cache values, about 250 KB in BF16, roughly 266 GB for a full 1M-token context.

Inference is supported in Transformers, llama.cpp, SGLang, vLLM, and MLX. The Transformers implementation provides MuseSparkForConditionalGeneration and MuseSparkProcessor, with DistributedConfig for tensor and expert parallelism; the BF16 checkpoint runs across 4 nodes of 8 H200 GPUs with TP=32, and NVFP4 runs on a single B200 node. The processor expands image markers, samples video at 2 FPS capped at 96 frames by default, resamples audio to mono 24 kHz and pads to 1920-sample multiples, and returns a mask so padded audio embeddings can be excluded. Reasoning effort is controlled by symbolic levels low/medium/high/xhigh mapped to 16/64/256/512, and reasoning cannot be turned off entirely. Tool calling works for text-only and multimodal messages and uses a channel-based chat template. llama.cpp quants have been tested on Apple Silicon; an imatrix-calibrated 2-bit quant runs at 18 tokens/s on an M3 Ultra. The SGLang reference runs the NVFP4 checkpoint across four GB300 GPUs, vLLM supports text/image with a 128K serving context on Blackwell, and the MLX oQ3e quant reaches about 225 tokens/s prefill and 21 tokens/s generation on an M3 Ultra. Hugging Face Inference Endpoints offers a validated SGLang deployment recipe for NVFP4.

For fine-tuning, TRL‘s SFTTrainer can train LoRA adapters on the attention projections while the experts remain frozen. The example uses 8 nodes of 8 H100s, with fsdp_size=64 and ep_size=8, packing examples into 2048-token sequences and using save_only_model because a sharded model cannot resume. The reported 500-step run took about 2 hours on 64 H100s at roughly 8,000 tokens per second, raising mean token accuracy from 72% to 84% and lowering loss from 1.26 to 0.57.

The post also shows three agentic demos: a 3D asset generator that combines a 2-bit quantized Muse Spark Open on llama.cpp with the Pi coding agent for iterative Three.js code generation from screenshots, a browser agent with OpenEnv and Hugging Face Sandboxes that acts on screenshots alone without DOM or HTML, and a voice-driven browser agent that searches Amazon based on spoken criteria.