Apertus 1.5 Adds Multimodal Inputs, Thinking Mode, and 262K Context

Apertus 1.5 extends the open-weight Apertus 1.0 family with native image and audio understanding, an optional thinking mode, a four-times-longer context window, and improved instruction-following and tool use. The updated 8B and 70B weights were released on Hugging Face on July 24, 2026 under Apache 2.0, and the project Technical Report (arXiv:2610.15542) documents the full pipeline. Both models are continued pretraining of the first generation: 4 trillion additional tokens for the 8B model and 2 trillion for the 70B model, bringing totals to 19T and 17T respectively. As with prior releases, weights, data, and training details are fully open.

Multimodality is introduced in an early-fusion manner: images and audio enter the model as discrete tokens in the same sequence as text, rather than through a separate vision encoder. Images are tokenized with the Emu3.5 tokenizer (a 512×512 image becomes 1,024 tokens) and audio with WavTokenizer (40 tokens per second). Because only the vocabulary is extended, the architecture is unchanged and training runs at text-only speed. The models can understand images and audio but generate only text.

Pre-training continues from Apertus 1.0 in four stages: unpaired text plus standalone images and audio, paired image–caption and audio–transcript data, a cooldown stage with more math and code, and a long-context stage that grows the window step by step from 8K to 262,144 tokens. More than half of the training tokens remain text-only to preserve language abilities, with refreshed sources including FineWeb2-HQ+, FinePDFs, and Nemotron-CC. All new data meets the same bar: publicly available, permissively licensed, with robots.txt opt-outs respected retroactively.

Post-training runs in five stages: SFT, long-context SFT, RL, DPO, and charter alignment via SDPO. SFT uses 3.7M text samples plus image and speech data; reasoning traces are added only for prompts the model cannot already solve, and tool-use data is restricted to execution-verified trajectories. Long-context SFT at the full 262K window restores abilities that short-context SFT erodes. RL with verifiable rewards (RLVR) covers about 350K prompts in math, code, instruction-following, multilingual, and vision, mixing thinking on and off, while DPO uses judge-scored preference pairs for chat quality, tool use, and safety. SDPO, applied to the 8B only, aligns the model with the Apertus Charter through judge feedback and self-distillation.

Evaluation scope expanded to include multimodal and thinking-mode benchmarks. Multimodal runs use the MLLM-eval-suite pipeline, merging lmms-eval and VLMEvalKit, while text evaluations use a public fork of lm-evaluation-harness. The release still trails frontier open-weight models, but Apertus 1.5 8B outperforms similarly sized open-weight models on instruction following and multilingual tasks, thinking mode improves performance on reasoning-heavy benchmarks such as AIME, and multimodal performance is closer to models like Gemma 4 12B, with competitive spatial and document/chart understanding. The 8B model is on par with leading open-weight models on Swiss Legal tasks and matches GPT-OSS-20B on the Last Translation Benchmark, outperforming Claude Haiku 4.5 and Google Translate.

Evaluation reporting is a major part of the release. Apertus collaborates with the EvalEval Coalition, publishing results in the Every Eval Ever (EEE) schema, which records who ran the evaluation, how the model was accessed, generation settings, and metric interpretation. This is the same infrastructure the UK AI Security Institute uses, and a dedicated converter lets anyone rerun the text evaluations. EvalEval has mapped EEE to governance requirements including Article 55 of the EU AI Act and the GPAI Code of Practice, finding the schema represents 56 of the 60 required evaluation-reporting elements.

Reproducibility materials include intermediate checkpoints (base, SFT, RLVR, DPO), curated data mixes, and full recipes in the Technical Report. Ablations show that mixing multimodal data with text during SFT helps across all modalities, and that benefits from multimodal reasoning traces survive subsequent text-only RL. Deployments already include Proton Lumo and Infomaniak Euria, with APIs announced by Swisscom, Infomaniak, and Phoeniqs. The team also previewed Apertus 2.0: a sparse/MoE architecture with roughly 600B total parameters, prioritizing agents, mathematics, and coding.

Apertus 1.5 Release Notes

View Original