Overview
The Linkup research team has released SPARSEUP, an open-source learned sparse embedding model built on a 149M-parameter ModernBERT backbone. The model is distributed under the Apache 2.0 license and is available for immediate deployment.
Linkup reports that SPARSEUP achieves an average nDCG@10 of 56.4 on BEIR-13, calling it the strongest public vocabulary-based sparse encoder under 150M parameters that the team is aware of. The model arrives at a time when open retrieval research is rapidly diversifying, with 2026 seeing renewed interest in sparse architectures that combine neural learned weights with the interpretability and efficiency of inverted indexes.
Deployability and Availability
Is it deployable? Yes. The weights are hosted on Hugging Face under Apache 2.0. The model can be loaded through either Transformers or Sentence Transformers using trust_remote_code=True. This makes it straightforward to integrate into existing retrieval pipelines without custom infrastructure.
Why a Sparse Model, and Why Now
Most open retrieval models are dense: they produce a single vector per text. Sparse models instead output weights over a vocabulary, where each dimension maps to a real token. This design offers several advantages:
- Vectors fit naturally into inverted indexes, enabling efficient keyword-style retrieval
- The representations are human-readable and interpretable
- They tend to match rare words and exact terminology more effectively
The catalyst for SPARSEUP was LightOn's DenseOn and LateOn release. LightOn published open data, a training recipe, a dense model, and a late-interaction model. SPARSEUP fills the missing sparse slot in that research ecosystem. By using the same backbone family and fine-tuning data, all three retrieval styles can now be compared side by side under consistent conditions.
How SPARSEUP Is Built
Training begins from the LateOn-unsupervised checkpoint. Because that checkpoint lacked an MLM (masked language modeling) head, the team grafted back ModernBERT's original one. Fine-tuning then proceeded using the same open data and recipe established by the LightOn release, ensuring methodological consistency across the dense, late-interaction, and sparse variants.
This build approach reflects a broader 2026 trend in the retrieval community: rather than training isolated models from scratch, researchers are increasingly reusing shared backbones and datasets to isolate the effects of retrieval architecture itself.
via MarkTechPost