Token Pooling-Aware Training for Multi-Vector Embedding Models
Multi-vector embedding models represent documents and queries as sets of token-level vectors, enabling fine-grained semantic matching. However, storing and searching over every token vector quickly becomes expensive as corpus size grows. Token pooling—reducing the number of vectors per document—offers a practical remedy, but models must be trained with pooling in mind to preserve retrieval quality.
Model Example
A concrete reference implementation is available on Hugging Face:
- Model:
tomaarsen/mve-ModernBERT-base-token-pooling - Task: Feature Extraction
- Parameters: ~0.1B
- Last updated: October 28, 2026
The repository demonstrates how a ModernBERT-base backbone can be adapted for multi-vector embeddings with token pooling-aware training. As of 2026, ModernBERT has become a common efficient backbone for embedding workloads, and practitioners increasingly expect pooling to be part of the training pipeline rather than a post-hoc compression step.
Why Token Pooling-Aware Training Matters
Token pooling reduces the number of vectors stored per document, which directly lowers index size and search latency. Yet pooling changes the representation the model is optimized against. Without accounting for pooling during training, retrieval performance can degrade noticeably. Training with pooling-aware objectives helps the model learn representations that remain discriminative after compression.
Practical Considerations
- Align the pooling strategy used at inference with the one simulated during training.
- Evaluate retrieval metrics before and after pooling to quantify any quality trade-off.
- Balance storage savings against accuracy when selecting pooling granularity.