Probe What-If Worlds Before You Ship: Persona Simulation for Real-World Decision Making

Authors: Julie Zhu, Jianheng Hou, Zhixu Silvia Tao, Shi Bo, Yilan Fan, Yixuan He, Zhiyuan Hu, Shirley Huang, Fangyu Liu, Yijun Wang, Zibu Wei, Haimingyue Xu, Zhen Xu, Zichen Yuan, Jierui Zuo, Jintao Huang, Yifan Wang, Qianfeng Wen, Jiahan Li, Zhejian Peng, Bowen Yang, Shushu Liang, Yuexing Hao, Xiaomin Li

Subject: Computer Science > Computation and Language (cs.CL)

arXiv: 2611.14700 — Submitted 10 Nov 2026

Abstract

User research and system testing reveal whether products and services meet user needs, yet recruiting specialized participants, designing controlled what-if comparisons, and repeating studies as designs evolve often bottleneck design iteration and data-driven decision-making. We present a framework and execution infrastructure that translates real-world industry inquiries into controlled experiments with simulated users. Our methodology formally defines the target cohort, experimental stimuli, and decision-relevant metrics, while the pipeline supports large-scale concurrent runs over cohort–stimulus–environment combinations (survey, web, chatbot, and OS app), auditable trial-level logging, cohort-level outcome aggregation, and cross-cohort and cross-variant statistical reports. Across twelve industry partner cases spanning five domains, we show how constructing hard-to-reach or what-if persona cohorts supports exploratory studies and distills simulated responses into testable design hypotheses.

Overview

As AI-driven product development accelerates in 2026, teams face mounting pressure to validate design decisions before shipping. Traditional user research—recruiting specialized participants, running controlled comparisons, and repeating studies as designs iterate—remains a persistent bottleneck. This paper introduces a framework and execution infrastructure that converts real-world industry inquiries into controlled experiments using simulated users.

Methodology

The proposed approach formally specifies three core components:

• Target cohort — the persona population under study, including hard-to-reach or hypothetical "what-if" groups.
• Experimental stimuli — the design variants or interventions being compared.
• Decision-relevant metrics — the outcomes that inform product and business decisions.

Execution Infrastructure

The pipeline supports:

• Large-scale concurrent runs across cohort–stimulus–environment combinations, covering survey, web, chatbot, and OS app contexts.
• Auditable trial-level logging for traceability and reproducibility.
• Cohort-level outcome aggregation for analyzing group behavior.
• Cross-cohort and cross-variant statistical reports to support comparative decision-making.

Industry Validation

Across twelve industry partner cases spanning five domains, the authors demonstrate how constructing hard-to-reach or what-if persona cohorts supports exploratory studies. The framework distills simulated responses into testable design hypotheses, enabling teams to probe alternative "what-if worlds" before committing to real-world deployment.

via ArXiv CL+LG