AI Augmented User Research: The New Rules for UX Teams

AI is transforming the way designers work, including the processes of conducting user research.

We have more opportunities than ever to capture user input: interviews, surveys, support tickets, analytics, reviews, usability tests, and session recordings. The true challenge lies in being able to quickly analyze this information and apply relevant insights to make better product decisions.

This is where AI can be leveraged to make our jobs easier.

However, one crucial consideration must guide our use of the technology: while AI can help us interpret evidence, it cannot replace actual users. If the technology suggests that a particular UI element may be confusing, this is only data. If multiple users report the same issue, this is research. These two pieces of information may be related, but they are not the same.

AI can help us become much more efficient in our research.

One reason we need to embrace AI assistance is that it can help reduce the amount of time we spend on research-related tasks that have nothing to do with actual users.

For instance, after conducting several interviews, an AI tool could potentially help us identify recurring themes, compare these themes across different segments of users, extract relevant quotes, and perform various other analyses to make it easier to understand what we learned.

This would free up more time for researchers and designers to focus on the parts of their work that truly matter: the insights. An AI tool may identify several themes, one of which is “control”, but it is for the researcher to interpret what this means for the user.

Does the user want more control over the product, or do they feel that the product takes too much control away from them?

Does the word “control” even resonate with them, or was it an unexpected finding that requires additional investigation?

While AI can help us find patterns, people must still give meaning to these patterns.

Research activities should become more continuous.

As products become more dynamic and more reliant on technologies such as AI, our ability to make frequent product updates is increasing.

This trend inevitably impacts the way we do research.

Instead of asking ourselves, “When should we do user research?”, we should be asking, “What can we learn from users this week?”.

AI can help make this type of ongoing research easier to implement, and, by extension, more common.

By enabling us to analyze smaller bodies of evidence more frequently, the technology can help make research an ongoing, continuous process.

AI can help us extract value from legacy research.

Most companies have a wealth of qualitative and quantitative research data stored in various digital repositories.

The information is often scattered, making it challenging to find relevant insights when the need arises.

By leveraging the power of AI, we can make legacy research much easier to navigate.

For instance, instead of asking, “What have our users said about our onboarding experience?”, a researcher may simply ask an AI tool to find relevant past research on the subject.

The researcher would then be able to review the existing transcripts, feedback, and analyses to identify relevant insights.

However, it is crucial that they continue to dig into the details to fully understand the context in which past users behaved the way they did.

While the AI tool can help us get to the evidence faster, it is up to people to actually engage with the evidence.

A summary is not always an insight.

As mentioned previously, an AI tool could easily summarize several user comments such as, “I usually skip onboarding.”, into a short statement reading, “Many users tend to skip onboarding.”

While this summary may be accurate, it may not contain much insight.

A researcher may realize that the reason many users skip onboarding is that they are already familiar with the product, but this would require them to dig deeper into the data to understand what users really mean.

Once the researcher gets to this point, they can identify an actual insight: users expect a different onboarding experience.

They may even conclude that the issue is not with onboarding itself, but rather the fact that it prevents users from doing what they want to do right away.

Synthetic users should be treated as such.

AI tools can generate synthetic users that can be used to test early-stage concepts and assumptions.

This capability can be invaluable in helping researchers prepare for actual research, such as by improving interview guides or identifying potential issues before engaging actual users.

However, synthetic users should not be used as a replacement for real users.

Actual users are unpredictable, and this unpredictability is part of the reason why they provide valuable insights in the first place.

Many researchers and designers fail to account for this characteristic when working with synthetic users, which can lead to poor assumptions and misleading conclusions.

AI can help us prepare for research, but it should not replace actual research.

Interviews moderated by AI can help scale qualitative research.

Various AI tools can help researchers conduct interviews and capture qualitative feedback from users.

This capability can be incredibly valuable when trying to obtain user input at scale.

It can help teams reach more users, especially if they are geographically distributed or working with non-English speaking users.

That being said, interviews conducted by a real person are sometimes necessary.

A researcher may detect nuanced non-verbal queues that could influence the interpretation of a user’s feedback.

In some cases, users may even give unexpected answers that can prompt a researcher to change the direction of an interview mid-stream.

These types of opportunities can be much harder to identify when an interview is moderated by AI.

This is not to say that AI moderation is useless; it is simply a tool that should be used judiciously.

The role of researchers and designers will evolve.

If AI tools help researchers organize and summarize evidence, their role will increasingly revolve around judgment.

Researchers will have to make better decisions about which evidence to prioritize, what to do when the evidence contradicts each other, and what to do when patterns cannot be identified.

They will have to engage more with the actual users, who are ultimately the reason why UX research exists.

The same trend will take place among designers.

While AI tools can generate design concepts with relative ease, it is ultimately up to designers to decide which concepts are actually worth pursuing.

As AI starts to dominate the mechanical parts of design, human judgment will become even more valuable.

A simple AI augmented research process

A research process that leverages AI assistance could look something like this:

• Define the research question.

• Use AI to review existing research and prepare for the study.

• Conduct the research with real users.

• Let AI summarize the results.

• Review the raw evidence before applying the findings.

• Apply human judgment to identify patterns and insights.

• Make product recommendations based on the findings.

AI can help organize and summarize research findings, but it is up to people to understand the implications of these findings.

The future of user research is human-centered.

While the idea of a fully automated research process may seem appealing at first, it ultimately detracts from the value of a human-centered approach.

The true potential of AI lies in its ability to make researchers more efficient and effective.

The technology can help us spend less time on tedious transcription and analysis tasks and more time engaging with users.

It can help designers find the insights hidden in legacy research reports.

It can help everyone make better product decisions based on the evidence at hand.

Finally, it can make research more accessible to smaller teams that do not have the resources to invest much time and effort into the process.

AI augmented user research should not make us lazier; it should make us better at our jobs.