NeoMME is a family of multilingual multimodal encoders (260M and 800M parameters) that processes both text tokens and raw image patches in a single bidirectional Transformer, without a separate pretrained vision tower or causal language model. The model is trained from scratch using a masked discrete-diffusion objective, where text tokens are corrupted at varying rates and the model must reconstruct them, optionally with visible image patches. This design is motivated by the observation that retrieval, classification, and token labeling do not require autoregressive generation, so the parameter and compute overhead of a causal decoder is unnecessary.

For visual document retrieval, NeoMME is fine-tuned into NeoMME-Retriever using the page-image methodology of ColPali. A forward pass produces both a dense embedding (mean pooling) and a late-interaction multi-vector embedding (128-dimensional per token/patch). On the ViDoRe v3 benchmark, NeoMME-Retriever-260M achieves an nDCG@10 of 0.523, the highest among models under 800M parameters, and comes within 0.002 of ColQwen2.5 while using about 14× fewer parameters. The 800M variant scores 0.556, within 0.009 of the similarly sized Vultron Retriever Flash. Both models lie on the Pareto frontier for nDCG@10 vs. model size.

High-resolution images produce large late-interaction embeddings—on average about 1.5 MB per page in float32. To reduce storage, NeoMME combines hierarchical token pooling (clustering similar vectors and replacing each cluster with its mean) with asymmetric quantization (documents quantized to int8 or binary, queries kept at higher precision). With a pooling factor of 8, int8 queries, and binary documents, storage drops to about 6 kB per page—a 255× reduction—while retaining more than 95% of the baseline nDCG@10. Users can choose a compression setting along the quality–storage frontier.

Throughput is measured on an NVIDIA L40S at 2048×2048 input: the 260M model encodes about 51 pages per second, roughly twice the throughput of ColModernVBERT. The 800M model is also faster than comparable models at smaller resolutions. NeoMME uses a 131k-token BPE tokenizer trained on multilingual text, code, mathematics, and machine-produced image transcripts. The architecture supports dynamic image resolution, a context length of 16,384 tokens, and attention that combines symmetric sliding-window layers with global attention every sixth layer and the final layer. The model is released under Apache 2.0 on Hugging Face Transformers, with separate checkpoints for dense and late-interaction fine-tuning via Sentence Transformers v6, or joint training via NeoMMEForRetrieval with a custom Trainer.