Token Clustering and Semantic Sequence Mamba for Hyperspectral Image Classification
Authors: Yimin Zhu, Mahmood Elahi, Lincoln Linlin Xu
arXiv: 2609.28580 [cs.CV]
Submitted: 23 September 2026
Subject: Computer Vision and Pattern Recognition (cs.CV)
Abstract
Although hyperspectral images (HSIs) provide rich spectral-spatial information, accurate pixel-level classification remains challenging because of spectral-spatial heterogeneity and complex spatial structures. Existing vision state space models (Mamba) typically construct sequences according to predefined spatial neighborhoods, without explicitly accounting for semantic similarity or spatial non-stationarity.
To address this limitation, we propose Token Clustering and Semantic Sequence Mamba (STMamba), which organizes sparse tokens into semantically coherent sequences for hyperspectral image classification. STMamba introduces the following key features:
- Macro level: A hierarchical encoder-decoder progressively selects semantic tokens with the Token Clustering Module (TCM) and restores dense features using a parameter-free Cross-scale Neighborhood Attention (CNA) Upsampler.
- Micro level: The TCM first identifies representative cluster centers through density-aware clustering and estimates soft memberships based on feature similarity. A quadtree-based dynamic selection strategy then retains sparse and spatially distributed tokens from each semantic cluster, forming coherent semantic-token sequences while reducing redundant pixel-wise representations.
- Sequence modeling: Parallel Spatial and Spectral Semantic-wise Sequencing Mamba (SWSM) modules capture complementary long-range spatial and spectral dependencies within homogeneous semantic token sequences while suppressing irrelevant interactions across heterogeneous regions.
Experimental results on three large-scale benchmark datasets demonstrate that STMamba outperforms state-of-the-art (SOTA) methods with respect to both quantitative and qualitative results.
Key Contributions
- Semantic-aware sequence construction โ Unlike conventional vision Mamba approaches that rely on predefined spatial neighborhoods, STMamba organizes tokens into semantically coherent sequences, explicitly accounting for semantic similarity and spatial non-stationarity.
- Token Clustering Module (TCM) โ Combines density-aware clustering, soft membership estimation, and a quadtree-based dynamic selection strategy to extract sparse, spatially distributed, and semantically representative tokens while eliminating redundant pixel-wise representations.
- Parallel SWSM modules โ Jointly model long-range spatial and spectral dependencies within homogeneous semantic regions, enhancing discriminative representation while suppressing cross-region interference.
- Parameter-free CNA Upsampler โ Enables efficient restoration of dense features from sparse semantic tokens without introducing additional learnable parameters.
Why It Matters in 2026
As hyperspectral remote sensing platforms continue to proliferate โ from next-generation Earth observation satellites to low-altitude UAV-based imaging systems โ the volume and spectral resolution of HSI data have grown rapidly. Efficient pixel-level classification under such conditions is increasingly critical for applications including precision agriculture, mineral exploration, urban land-cover mapping, and environmental monitoring.
STMamba arrives at a moment when state space models have become a dominant paradigm for efficient long-sequence modeling in vision. By integrating semantic clustering with Mamba-based sequence modeling, this work addresses a key bottleneck in the field: the mismatch between predefined spatial scanning orders and the inherent semantic heterogeneity of hyperspectral scenes. The approach aligns with the broader 2026 trend toward semantic-aware and content-adaptive sequence modeling, offering a promising direction for scalable HSI analysis.
Cite as: arXiv:2609.28580 [cs.CV]
DOI: https://doi.org/10.48550/arXiv.2609.28580
via ArXiv CV