Imbalanced Data Clustering via Targeted Data Augmentation Using GMM and LLM

In Natural Language Processing (NLP), the presence of underrepresented topics poses significant challenges, particularly in unsupervised learning tasks where clustering algorithms often fail to adequately capture minority classes. To address this issue, we introduce a novel unsupervised data augmentation method that integrates Gaussian Mixture Models (GMMs) with Large Language Models (LLMs). GMMs, known for their flexibility and robustness, are employed to identify clusters corresponding to underrepresented regions within the data, while LLMs generate synthetic documents to enrich these clusters and enhance their representation. Through experiments on various imbalanced text datasets, we demonstrate that our approach not only preserves clustering performance across all cases but frequently improves cluster interpretability. This offers a robust and scalable solution for enhancing data representation in unsupervised NLP tasks, particularly relevant in 2026 as the demand for handling real-world, skewed data continues to grow.

via ArXiv CL