Overview
NCP-ArchPreview is a technical report from the Intern-NCP Team (arXiv:2609.10715, submitted 9 September 2026) presenting a latent-space language model that extends autoregressive pretraining beyond the conventional next-token prediction paradigm. The work centers on Next Concept Prediction (NCP), a training objective that moves language modeling into a continuous, concept-level latent space rather than operating solely over discrete tokens.
Motivation: Beyond Next-Token Prediction
Standard large language models are trained by predicting the next token in a sequence. While effective, this objective forces the model to reason through a discrete, human-designed vocabulary and can spread meaning across many tokens. NCP-ArchPreview explores an alternative: learn and predict at the level of concepts in a latent space, aiming for more compact and semantically grounded representations of language.
Key Idea: Next Concept Prediction
The core contribution is the NCP training objective, which reframes prediction as operating over latent concepts instead of surface tokens. This positions the model as a step toward latent-space language models (LSLMs) β systems that reason and generate within a continuous conceptual representation rather than a purely token-based one.
Context in 2026
The report arrives amid growing interest in latent reasoning and token-free architectures. As of 2026, the field has been actively probing alternatives to token-level autoregression, including latent chain-of-thought, continuous reasoning spaces, and concept-level pretraining. NCP-ArchPreview contributes to this trajectory by offering an architectural preview of a concept-prediction-based language model.
Authors and Affiliation
The work is a collaborative effort by the Intern-NCP Team, with contributors including Jiaqi Cao, Chiyu Chen, Shuang Cheng, Xu Cheng, Beiya Dai, Yufan Feng, Kewen Ge, Ruijun Ge, Jiayi Huang, Yang Jiao, Dahua Lin, Zhouhan Lin, Yifan Liu, Yuliang Liu, Biqing Qi, Mowen Ruan, Junzhe Shen, Yunchong Song, Hao Sun, Zhongbo Tian, Yixuan Wang, Rubin Wei, Jiaxin Xiong, Kangyu Yang, Qian Yao, Qi Zhang, and Bowen Zhou.
Significance
By targeting concept-level prediction in latent space, NCP-ArchPreview signals a shift in how autoregressive pretraining may be designed and evaluated β moving from token-centric pipelines toward architectures that model meaning more directly. The technical report serves as an architectural preview rather than a finalized system, inviting further research into latent-space language modeling.
Reference: The Intern-NCP Team, "NCP-ArchPreview Technical Report: Moving towards Latent Space Language Models through Next Concept Prediction," arXiv:2609.10715 (cs.CL), 9 September 2026.
via ArXiv CL+LG