Synthetic Users in UX Research

There is something very tempting about the idea of synthetic users.

They are fast. They are cheap. They scale easily. They can generate polished answers in seconds and make it feel as if you have just compressed weeks of research into a single afternoon. For teams under pressure, that promise is hard to ignore.

Recent research suggests they may be useful in some narrow situations, especially in early ideation, piloting, or rough exploration. But the same body of work also shows that they can distort behavior, flatten human messiness, and create a false sense of confidence if they are treated like a substitute for real users.

Why synthetic users sound so appealing

The appeal is easy to understand. Synthetic users seem to offer a way around some of the hardest parts of UX research: recruiting participants, waiting for data collection, paying incentives, scheduling interviews, and dealing with limited timelines. In theory, they can help researchers test ideas earlier, explore possible user reactions, and move through the messy front end of design more quickly.

And to be fair, some of the literature supports that limited use. A number of papers suggest synthetic users can help with brainstorming, rough persona generation, and certain kinds of early stage exploration. In those cases, they may be useful as an aid to thinking, not as evidence.

The real problem is not whether they sound believable

The real problem is that believable is not the same as valid.

One of the clearest themes in the literature is that synthetic users often generate responses that sound plausible, coherent, and even insightful. But that surface quality can be misleading. What they produce is not human experience. It is a prediction shaped by patterns in training data.

That means synthetic users may tell you what a user is expected to say, not what a real user would actually do in context.

In UX, that difference is huge.

Real users contradict themselves. They misunderstand instructions. They invent workarounds. They quit halfway through tasks they claimed they cared about. They tell you one thing and do another. They bring frustration, habit, confusion, emotional baggage, and situational context into every interaction. That messiness is not noise. Very often, it is the signal. Synthetic users tend to smooth that away. And once that happens, the research may look cleaner while becoming less useful.

What the literature is actually showing

The evidence so far does not support a simple yes or no answer.

On one side, some studies showed that LLM generated HCI style responses could appear believable, but even there the conclusion was cautious: such outputs might help with ideation or piloting, yet findings still need validation with real data. A later systematic review reached a similar conclusion, arguing that AI based synthetic users may accelerate early exploration while also raising concerns about authenticity, variability, bias, and ethical validity.

On the other side, some of the stronger critical work argue that synthetic users struggle with novelty and can reflect normative bias, which makes them especially weak for tasks like interviews and usability testing.

The biggest risk: false confidence

The danger with synthetic users is not just bad data.

It is false confidence wrapped in polished language.

Messy responses often make researchers cautious. Clean responses can do the opposite. They can make a weak conclusion feel stronger than it is. They can create the illusion that a finding has been validated when in fact it has only been simulated.

This is especially risky in product environments where teams are already under pressure to move quickly, show impact, and cut research costs. Synthetic users fit that pressure very well. They deliver something that looks like research, sounds like research, and can be presented like research. But if the underlying process is not grounded in real user evidence, the output may be convincing without being trustworthy.

That is a dangerous place for UX to go.

So, should UX researchers use synthetic users?

My answer is yes, but only with clear limits.

Use them for rough exploration. Use them to generate possibilities. Use them to support internal thinking. Use them when you need a starting point, not when you need evidence.

But when the goal is to understand real behavior, uncover unexpected pain points, evaluate usability, or make decisions that affect actual people, synthetic users should not replace research with real participants.

They are not users.

They are generated approximations of what users might say.

And that is a very different thing.

Final thought

The future of UX research will absolutely include AI. That part is obvious.

Synthetic users may have a role, but only if we stay honest about what they are and what they are not.

Because in UX, a response that sounds human is still not the same thing as insight grounded in human reality.

References

Hämäläinen, P., Tavast, M., & Kunnari, A. (2023, April). Evaluating large language models in generating synthetic hci research data: a case study. In Proceedings of the 2023 CHI conference on human factors in computing systems (pp. 1-19).

Pehar, F. (2025, June). AI as Synthetic Users in Human-Centred Design: A Systematic Review. In 2025 MIPRO 48th ICT and Electronics Convention (pp. 1500-1509). IEEE.

Salminen, J., Amin, D., Jung, S. G., & Jansen, B. (2025, March). The Use of Large Language Models in HCI: A Critical Analysis of Synthetic Users. In Proceedings of the Augmented Humans International Conference 2025 (pp. 413-417).

Xiang, W., Zhu, H., Lou, S., Chen, X., Pan, Z., Jin, Y., ... & Sun, L. (2024, May). Simuser: Generating usability feedback by simulating various users interacting with mobile applications. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (pp. 1-17).

Schuller, A., Janssen, D., Blumenröther, J., Probst, T. M., Schmidt, M., & Kumar, C. (2024, May). Generating personas using LLMs and assessing their viability. In Extended abstracts of the CHI conference on human factors in computing systems (pp. 1-7).

Shin, J., Hedderich, M. A., Rey, B. J., Lucero, A., & Oulasvirta, A. (2024, July). Understanding human-AI workflows for generating personas. In Proceedings of the 2024 ACM designing interactive systems conference (pp. 757-781).

Zhong, R., McDonald, D. W., & Hsieh, G. (2025). Synthetic Cognitive Walkthrough: Aligning Large Language Model Performance with Human Cognitive Walkthrough. arXiv preprint arXiv:2512.03568.