Research is more than a conversation

How understanding human behavior can transform product development

Illustration by Eirian Chapman

I've been in this field for 22 years and I've thought often about what it is that makes what we do “research.”

One of the biggest rewards for researchers is developing close partnerships with the product teams we support, and facilitating this culture of dialogue with customers. But it’s worth asking: What are we as researchers uniquely responsible for delivering? And how does defining our successes distinguish us from everyone else on a product team who "talks to customers"?

What are researchers uniquely responsible for delivering?

There are four criteria for good research: Is it accurate, with data that is true and reasonably free from bias? Is it predictive, gathered and synthesized in such a way that findings can be extended to a larger audience? Is it actionable, and point to clear steps or changes that the team can implement in a feature or product? And is It insightful, pulling out non-obvious but essential findings that help change the underlying models of the product team’s thinking?

These criteria are a great way to start understanding the value a research team brings to the development process. But good research also relies on a researcher’s ability to work closely with the product team to develop a holistic understanding of the background, goals, and context of the product in the market.

To conduct good research, we must develop a deep understanding of the problem. It’s not enough to ask the product team what research questions they have. Our job is to understand how those questions relate to the business, the product strategy, and the assumptions being made by the product team that may or may not be accurate. At times I've likened this relationship to a marriage therapist whose goals are to understand both parties—in this case, the business goals of the product team and the user needs of the customers—to uncover the natural affinity between them.

Once the product team’s goals and constraints are well-understood, a good researcher must identify which methods will be most effective to learn about the problem. That means bringing to the table our deep understanding of psychology and human behavior, and removing as much bias from the process as possible to achieve accurate and predictive results. The foundation of cognitive psychology is based on the difference between what people say they do and what they actually do, which means that asking direct questions doesn’t always result in accurate answers.

As an example, newer users consistently ask for more help onboarding and learning our products. After several studies during which we watched users onboard to creative products, we observed a consistent pattern: Users begin their learning journey by “orienting” themselves to the product through play behavior (most notably trying out tools familiar to them from other applications and comparing them with our implementations). This play behavior helps people develop a mental model for the application that’s necessary for them to process and understand the learning options afforded to them. That means that any product tours or guidance prior to this self-orientation is going to be ineffective, since they’re presented at the wrong time, in the wrong context, and often with the wrong level of detail to be truly helpful.

How do we set ourselves apart from other team members who “talk to customers”?

Qualitative data requires understanding hidden motivations and unmet needs that, if they were obvious, wouldn’t need anyone to research them in the first place. Our job is to continually go beyond products being “hard to learn” and supply data to support new learning methods, product changes, or marketing shifts to ensure that our products get into the hands of the people who have a high motivation to learn them.

As researchers, these deeper insights come from mixed methods of observing and asking questions and listening to users in ways that most people don’t.

I did a study many years ago to examine personal file organization for creative professionals. Most users reported relying heavily on system-wide searches to find files on their computer but, in fact, users almost universally searched for folders not files. This realization completely changed our interpretation of the self-reported data we had. In this study we avoided asking users to describe how they accessed their content, and instead set up a scenario in which they were invited to show us a recent project they were working on. They were not aware that our aim was to watch their navigation but in doing so, we quickly realized that there was a big disconnect between what users ask for and what would be valuable to them.

Often our work is nuanced and what we deliver goes beyond a simple reporting of data: Our goal is to communicate understanding, empathy, and to help our stakeholders get inside the user’s mind. Good research must go beyond reporting what people did, and instead allow us to develop models that help us predict how users might behave in similar situations. We study populations of people, and develop an expert understanding of their goals, workflows, and unmet needs. Bodies of knowledge that come from doing multiple studies on the same population over time cannot be distilled into a single report, so our job is to create a foundation of understanding that we can carry forth as expert guides to understanding people.

Researchers rely heavily on their relationships with the product team and involvement in product decision making to help them assess often subtle or unspoken assumptions and goals for the product. Just as we seek out subtleties in a customer’s motivations, we similarly use our research mindset to identify points of misalignment and differences in understanding across the product organization. A great relationship between a researcher and a product team can create a broad canvas for reframing problems to support innovation.