How to Analyze Changes in User Attitudes Over Time: A Practical Guide for UX Research

Most UX research still treats user experience as a series of snapshots. We run a usability test, collect a System Usability Scale score, send a Net Promoter Score wave, or run an intercept survey at the end of a flow. These are useful for diagnosing immediate friction and capturing static sentiment, but they miss something fundamental about how humans interact with products over time.

User attitudes are not fixed states. They are trajectories. Trust builds, then cracks. Frustration accumulates and sometimes diffuses. Competence grows and then plateaus. Real user experience is shaped by cumulative interactions, learning curves, feature rollouts, team processes and broader market forces. If we treat UX as a static state, we will always be late to the problem.

The next frontier in UX is not a new score. It is a shift from state based measurement to process based modeling. This piece walks through a toolkit for doing that in a rigorous way. It combines ideas from psychometrics, econometrics and machine learning, and translates them into concrete methods you can use to track and model user attitudes over time.

We will look at:

• Continuous time structural equation modeling to handle irregular usage and feedback

• Latent transition analysis to mathematically formalize user journeys and adoption stages

• Bayesian hierarchical longitudinal models for small samples and B2B contexts

• Joint models and Hidden Markov Models to connect what people say with what they do

• Modern data collection like Computerized Adaptive Testing, conversational surveys and passive sensing

• New metrics that move beyond satisfaction toward digital flourishing and perceived competence

• Practical issues like attrition, inverse probability weighting, method selection and real world practices from companies like Spotify

The goal is simple. Turn longitudinal UX from a buzzword into a practical, defensible practice.

Why Static UX Metrics Are Not Enough

Traditional UX metrics like SUS, NPS or one off CSAT items are almost always collected at a single point in time. This cross sectional approach quietly assumes that the attitude you capture at time t somehow represents an enduring state. In reality, user sentiment is highly volatile. It is shaped by recency effects, context, mood, workload and the specific path the person just followed.

A user can report high satisfaction right after a successful task, but have their trust erode over the next few weeks because of minor frictions they never bother to report. Static measures see the high point and miss the downward slope. They tell you what the attitude is, not how fast it is changing or whether that change is accelerating.

Cross sectional designs also make causal interpretation almost impossible. If you observe that people who are highly engaged also report higher satisfaction, you still do not know which caused which, and you have no way to establish temporal ordering or rule out rival explanations. Longitudinal designs are the only real way to disentangle these chains.

On top of that, snapshot metrics like NPS are inherently transactional. They compress complex, evolving relationships into a single number that is easy to track but blind to trajectories. If you care about relationships rather than transactions, you need methods that follow people over time.

Modeling Attitude Change in Real Time: Continuous Time SEM

One of the most powerful approaches for tracking how attitudes change is Continuous Time Structural Equation Modeling, often implemented through ctSEM. Unlike traditional models that assume you measured everyone at regular, fixed intervals, continuous time models treat time as it actually unfolds. They model latent processes as evolving continuously, and then use the exact time gaps between measurements in the estimation.

In practice, this means you no longer pretend that “survey 1” and “survey 2” are always one week apart. If one user responds at 9 in the morning and again at 5 in the afternoon, and another responds at 9 on Monday and then 9 on Wednesday, those time gaps are treated differently and the model scales its parameters by the actual time difference. That is crucial if you run diary studies, experience sampling, event triggered surveys or have feedback tied to usage rather than fixed waves.

For UX, this is a big shift. Think of a streaming platform that prompts for content enjoyment after some viewing sessions. Some users watch daily, others watch weekly, others go through phases of binge and silence. A traditional discrete time model treats each next rating as the same unit of time and therefore cannot distinguish between enjoyment stability over 8 hours versus 30 days. A continuous time model can. It can estimate drift (how strongly enjoyment tends to return to a personal baseline), diffusion (how volatile enjoyment is) and cross effects (for example, how usability perceptions feed into satisfaction over time and vice versa).

With enough repeated measurements, you can use continuous time SEM to predict exactly when enjoyment or trust is likely to drop below a critical threshold for a given user segment, and trigger interventions such as content recommendations or educational nudges at the right moment. In short, you move from “we know the average satisfaction at three time points” to “we understand the dynamics of satisfaction and trust in real time”.

There are costs. These models are more complex to learn and implement, they usually require specialized software and can be computationally expensive, especially if you attempt very large latent structures or Bayesian versions. They work best when you have at least three measurement points per user, and ideally ten or more. When you are in that regime, they are a very natural fit for mobile sensing, ESM and in the wild product feedback.

Not Every User Evolves the Same Way: Latent Transition Analysis

Personas and segments are a staple of UX, but they are almost always static. Someone is tagged as a novice, a power user, a skeptic, a promoter. What is usually missing is a picture of how people move between those states over time. Latent Transition Analysis is designed for exactly that problem.

LTA is a longitudinal extension of latent class analysis. It identifies latent classes based on observed indicators, then estimates the probability that someone will move from one class to another between time points. You can think of it as a mathematical version of a user journey map. Instead of sketching stages like Awareness, Consideration, First Use, Adoption, Loyalty, you let the model infer the stages from behavior and responses, then quantify the probability of progression and regression.

This is particularly useful for tracking adoption barriers and engagement states. For example, a product might end up with latent classes that correspond to basic users, occasional collaborators and power coordinators. LTA will not only tell you how many people are in each class at each time, it will tell you how likely it is for a basic user to become a collaborator during the next month, and how likely it is for a collaborator to drop back into basic use.

If you add predictors, such as whether someone completed a specific onboarding tutorial, attended a training or used a contextual help feature, you can see how those interventions change the transition probabilities. Perhaps basic users have only a five percent chance of becoming collaborators in a typical month, but users who see an interactive tooltip in the right context have a fifteen percent chance. That is direct evidence that design changes are not only moving metrics, they are changing how people move across states.

LTA is powerful, but it is data hungry. You generally need large samples to estimate transition matrices reliably, especially if you have many classes or rare states. You also need categorical or ordinal indicators measured at multiple time points. Choosing the number of classes involves a mix of fit statistics and judgment. When the data and design fit, LTA turns vague persona talk into quantifiable pathways and helps you spot who is stuck, who is at risk and where the journey breaks.

What If Your Sample Is Small: Bayesian Longitudinal Modeling

A lot of UX contexts do not look like consumer apps. They look like enterprise dashboards, clinical tools, manufacturing systems, learning platforms or accessibility interfaces with a small, specialized user base. In those cases, you might have twenty participants in an onboarding study or fewer than ten in a screen reader experiment. Traditional longitudinal models either fail to converge or produce unstable estimates.

Bayesian hierarchical longitudinal models are built for this situation. They combine prior information with observed data and treat repeated measurements as nested within individuals, who in turn are nested within segments or roles. Each user has a personal trajectory, but those trajectories are partially pooled toward a group level model, which allows the method to “borrow strength” across users.

In practice, this means you can make reasonable inferences about change even when you have few data points per user and a small number of users overall. You can also use prior knowledge, for example from a previous release or from similar products, to stabilize estimation. Bayesian methods do this naturally through prior distributions, which is very different from plugging in a “correction” after the fact.

This approach is especially important when the average hides real divergence. A redesign might improve perceived efficiency for new employees, while slightly reducing efficiency for seasoned experts. The average curve looks flat, so a naive analysis would conclude that nothing changed. A Bayesian hierarchical model will often reveal two distinct patterns, for instance a group whose efficiency is clearly increasing and a smaller group whose efficiency is stagnant or declining. It will also provide credible intervals that you can interpret directly, such as “there is a ninety five percent probability that efficiency is improving for this subgroup”.

Bayesian approaches also handle missing data more gracefully than simple listwise deletion. If someone skips a survey or misses a week of data, the model does not throw them away, it imputes likely values based on their own past pattern and the wider group. The main trade offs are computational cost and sensitivity to the choice of priors when samples are extremely small. But in B2B, accessibility or specialized UX research, Bayesian models open doors that classical methods simply cannot.

Bridging What Users Say and What They Do

Collecting better longitudinal survey data is only half the story. Modern UX data is a blend of telemetry and self report. People click, scroll, rage click, idle, churn, renew and upgrade. They also give ratings, write comments and respond to diary prompts. If you treat these two streams separately, you will miss the way that attitudes drive behavior and behavior influences attitudes.

Two families of methods help bridge the gap.

The first is joint modeling. Joint models estimate a longitudinal process such as satisfaction, trust or motivation and a time to event process such as churn, upgrade or subscription renewal at the same time. The two processes are linked through shared random effects. This is crucial because you stop observing satisfaction when someone leaves. If you ignore that informative dropout, you get biased estimates. Joint models explicitly treat the satisfaction trajectory as informative for the risk of churn and vice versa.

In a UX setting, you might track weekly motivation for a fitness app and connect it to subscription renewals. A joint model can reveal that not only low motivation, but high volatility in motivation is a strong predictor of churn. Users who swing between high and low motivation are more likely to cancel than those who are consistently moderate. That insight changes how you design interventions, because smoothing volatility may matter more than just pushing average scores up.

The second family is Hidden Markov Models and related state space models. HMMs assume there is an underlying, unobserved state such as confusion, flow or disengagement that generates the behavior you see. Click patterns, navigation sequences, help page visits, error events or even mouse movement become emissions from that hidden state. If you have periods where both survey labels and behavior are observed, you can train an HMM to decode state directly from telemetry in the periods where you do not ask questions.

For UX, that means you can estimate frustration levels purely from interaction patterns, identify states like “stuck explorer” or “goal directed power user”, and understand how people move between those states. You can see, for example, that “search followed by help page followed by back to search” is often a path through confusion, while “search followed by filter followed by checkout” corresponds to a focused, high intent state. Companies like Uber and Airbnb have developed internal tools that explicitly visualize these kinds of paths and state transitions, which moved them beyond static funnels into more realistic descriptions of user journeys.

Dealing With Attrition and Bias

No discussion of longitudinal methods is complete without attrition. People drop out, and they rarely drop out at random. Dissatisfied users churn. Overworked participants stop answering. If you analyze only the people who stay, you will often see an artificial improvement.

Inverse probability weighting is one robust way to address this. You model the probability that a given user would stay in the study based on their past data and traits, then weight the remaining users by the inverse of that probability. Users who look like they should have dropped out but did not effectively stand in for those who left. This does not magically fix every problem, but it is a necessary step if you want your longitudinal estimates to be valid.

Choosing the Right Method for the Job

With all these tools, method selection becomes a design problem in itself. Some rough pairings often work well:

• If you have high frequency mobile usage, irregular timestamps and rich telemetry, continuous time SEM is a good fit.

• If you care about stages of adoption or churn, and you can collect categorical indicators over time at scale, latent transition analysis will let you model user journeys as state transitions.

• If you work in B2B or accessibility contexts with small samples and care about individual trajectories, Bayesian hierarchical models are your friend.

• If you want to connect attitudes to outcomes like churn or upgrades, joint models are ideal.

• If you want to infer hidden emotional states from behavior, Hidden Markov Models and state space models are a strong option.

f you’re interested in building longitudinal UX systems that predict user behavior and emotional change, feel free to reach out at admin@puxlab.com. We help teams replace static surveys with scientific models that capture what users are becoming, not just what they say today.