Validation vs. Discovery Interviews

As a founder or product manager trying to learn the ropes of user research, someone will inevitably tell you to write a discussion guide. This is an unavoidable fact.

This is terrible advice. Discovery research aims to understand people’s needs so that the right solution can be delivered. Yet trained researchers will prescribe you a discussion guide irrespective of your research objective. Oh, the irony :))

In this essay, I’ll cover three key reasons why discussion guides tend to backfire for product managers (and pre-product founders) conducting discovery research:

Okay, let’s start by covering the basics…

Discovery research is the search for unexpected insights into your customer’s life. It’s used to figure out what we should build (product discovery) before we build it (product delivery).
Because we’re researching things that haven’t been built yet, discovery focuses on uncovering target customers’ unmet needs, pains and desires — not validating feature ideas, roadmap tickets, or wireframe mockups.
There are a bunch of different types of research that are used for discovery but this essay will focus on user interviews — the most commonly used method for discovery research.
A discussion guide is the list of questions you intend to ask during a user interview or focus group.

Seems simple, right?

… right???

Problem 1: Validation Tunnels vs. Discovery Deltas

Validation and discovery have opposite objectives.

Validation means you’ve already identified your intended destination and you’re looking to confirm that it’s the right place to go. Discovery is more about the journey than the destination — you’re pretty open-minded about where it could take you and what kind of unexpected treasures you might find along the way.

I like to imagine validation as a tunnel and discovery as a river delta.

The Validation Tunnel has a binary outcome — you either succeed and come out the other end or you fail and return where you started. It’s claustrophobic and very narrow in scope.

The Discovery Delta is the exact opposite. It’s sprawling and full of possible pathways — some of which branch off into many new directions, while others even link back to points you passed earlier. You’re not trying to get to one specific place, you’re just exploring the fertile channels with an open mind.

Most PMs already know this. They acknowledge that they’re exploring the Discovery Delta, not rushing through the Validation Tunnel. But an in-depth discussion guide is like committing to a set of directions upfront — your list of pre-planned questions will tie you to a specific route through the delta, preventing you from pursuing unexpected insights along the way.

Here’s a great example from Rob Fitzpatrick’s book The Mom Test of what it looks like to be stuck in the Validation Tunnel during a user interview:

Founder: “We’re building phone and tablet apps to help people stay in shape and just wanted to understand how you stay fit.”
Interviewee: “Okay.”
Founder: “How often do you go to the gym?”
Interviewee: “Not really ever.”
Founder: “What would you say is your biggest problem with going to the gym?”
Interviewee: “I guess the time it takes to get there. I’m always kind of busy, you know?”
Founder: “Perfect. That’s great. And could you rank these 4 in terms of which is most important to you in a fitness program: convenience, personalization, novelty, or cost?”
Interviewee: “Probably convenience, then cost, then personalization, then novelty.”
Founder: “Okay. Awesome. Thanks so much. We’re working on an app to help you exercise in the convenience of your own home. I think it’s going to be a great fit for what you care about.”

The founder here is so clearly tunnel-vision focused on validating their idea. Nothing can get in their way of successfully validating their product idea. Regardless of the answer they receive, this founder will just continue listing off their pre-planned questions.

As Rob points out in the book, this interviewee’s problem is not the inconvenience of the gym — pushups are pretty convenient, but the interviewee doesn’t do pushups. Or maybe he does? The founder didn’t bother asking what they do for exercise outside of the gym. So I guess we’ll never know.

The Validation Tunnel is filled with rejected founders and product managers wondering why nobody will be their customer. It’s a sad place to be — I know from experience. Save yourself the hassle and explore the Discovery Delta instead.

Problem 2: The Blank Page Challenge

The challenge with discussion guides isn’t usually that you can’t figure out the right questions to ask — it’s that you have too many questions to ask in the first place.

“What is a customer interview? We tend to think about it as; ‘I need to have a three-page discussion guide and it's going to be an hour long and it's this really formal research activity.’ But I actually think we can do five-minute customer interviews; it just depends on what we're trying to learn.” — Teresa Torres, author of Continuous Discovery Habits

The blank page challenge in discovery research is the opposite of writer’s block. Rather than agonizing over an empty page all day, you’ve likely got 101 potential interview questions brainstormed in under 15 minutes. The challenge becomes keeping the list of questions short rather than sprawling and unmanageable.

But why is this a problem at all? Surely having a longer list of questions gives you a better chance of finding a golden nugget insight during your interview?

The root of this problem is a misunderstanding of your role as a discovery researcher. Your primary objective isn’t to ask great questions, it’s to listen for moments that surprise you and then pull on those loose threads. If you’re immediately planning your next question, then you’re doing it wrong.

Will the discussion guide be helpful once that loose thread has been identified? Nope! At that stage, you don’t need any fancy research methods — you just need to ask ‘Why?’ a couple of times.

But this still begs the question; what should my first question be in order to get them talking in rich detail about a specific memory? And what memory should I ask them to focus on?

This is the real challenge of discovery research — articulating your research objective.

If you’re a pre-product founder, your discovery research is likely focused on a specific occasion, use case or customer objective. You want to zoom in as much as possible on this singular activity with a homogenous sample of target customers to learn about what they’re trying to accomplish, why that’s important to them, and what’s getting in their way.

If you’re part of a product team doing discovery research, you’ll want to start by first understanding what company objective you’re trying to impact and then map out the opportunity areas (unmet needs, pains, desires) underneath that (ie. the OST model). Taking ‘increase activation rate’ as an example company objective → dig into the experience new sign-ups have with your product to identify the barriers limiting their success.

In either case, you’ll want to articulate the focus of your research and then let your interviewee spend +80% of the time just talking about their personal experience. Your job is to give them space to speak from their perspective, watch out for unexpected insights, and show some genuine curiosity.

Problem 3: Wrong Side of the Sandwich

There’s a reason why I’ve been pretty binary about discussion guides for discovery research throughout this essay. We’ve been missing one important point of nuance that needs to be explained…

Discovery interviews are completely different depending on whether you’re at the breadth or depth stage of the process. That’s where The Discovery Sandwich comes in. It’s a simple framework for splitting discovery research into three clear steps:

Breadth: The first step identifies the various needs, pains and desires your customer feels when they’re trying to complete the ‘activity of focus’ (ie. the occasion / objective / use case you’re focused on).
Prioritization: The middle step takes all your breadth insights and quantitatively ranks them based on the ranking criteria you choose (eg. frequency, impact, or importance of the problem). This is based on Shreyas Doshi’s framework for Customer Problem Stack Ranking, which explains how solving a customer problem isn’t enough to guarantee success — you need to validate that the problem is a high-priority hair-on-fire pain point and not just a mild inconvenience for customers.
Depth: The final step carries forward two key data points from the previous step: (i) the highest-ranked pain point and (ii) the participants that personally ranked that pain point as their top priority. You can then dig deep into the details with these pre-screened participants without worrying about biasing the conversation by focusing on the pre-validated pain point.

You don’t even need a discussion guide for the Breadth step of the Discovery Sandwich. Just ask them to talk about a specific recent time that they tried to accomplish the key activity you’re researching.

A discussion guide can be useful in the final stage of the Discovery Sandwich where you want to get really deep into the details and context that surrounds the specific pain point you’ve validated as highest priority. But the same logic applies — your job as a researcher is still to find and pull on loose threads that lead to unexpected insights, not to fill the interview with pre-planned questions.

What to do instead of writing a discussion guide

If you’re at the start of a discovery research project, you’re better off answering these three questions instead of diving into discussion guide mode:

What is your activity of focus? This should be the occasion your target customer finds themself in, the use case they are trying to fulfill, or the objective they’re working to accomplish.
When was the last time your interviewee attempted this activity? Ask them to walk you through that specific experience from the very beginning — the moment they decided to pursue it.
Why? Whenever they say something that surprises you, ask why they did that.

That’s everything. Time for you to go find some loose threads to pull :)

Daniel