Validation vs. Discovery Interviews

As a founder or product manager trying to learn the ropes of user research, someone will inevitably tell you to write a discussion guide. This is an unavoidable fact.

This is terrible advice. Discovery research aims to understand people’s needs so that the right solution can be delivered. Yet trained researchers will prescribe you a discussion guide irrespective of your research objective. Oh, the irony :))

In this essay, I’ll cover three key reasons why discussion guides tend to backfire for product managers (and pre-product founders) conducting discovery research:

Seems simple, right?

… right???

Validation and discovery have opposite objectives.

Validation means you’ve already identified your intended destination and you’re looking to confirm that it’s the right place to go. Discovery is more about the journey than the destination — you’re pretty open-minded about where it could take you and what kind of unexpected treasures you might find along the way.

I like to imagine validation as a tunnel and discovery as a river delta.

The Validation Tunnel has a binary outcome — you either succeed and come out the other end or you fail and return where you started. It’s claustrophobic and very narrow in scope.

The Discovery Delta is the exact opposite. It’s sprawling and full of possible pathways — some of which branch off into many new directions, while others even link back to points you passed earlier. You’re not trying to get to one specific place, you’re just exploring the fertile channels with an open mind.

Most PMs already know this. They acknowledge that they’re exploring the Discovery Delta, not rushing through the Validation Tunnel. But an in-depth discussion guide is like committing to a set of directions upfront — your list of pre-planned questions will tie you to a specific route through the delta, preventing you from pursuing unexpected insights along the way.

Here’s a great example from Rob Fitzpatrick’s book The Mom Test of what it looks like to be stuck in the Validation Tunnel during a user interview:

The founder here is so clearly tunnel-vision focused on validating their idea. Nothing can get in their way of successfully validating their product idea. Regardless of the answer they receive, this founder will just continue listing off their pre-planned questions.

As Rob points out in the book, this interviewee’s problem is not the inconvenience of the gym — pushups are pretty convenient, but the interviewee doesn’t do pushups. Or maybe he does? The founder didn’t bother asking what they do for exercise outside of the gym. So I guess we’ll never know.

The Validation Tunnel is filled with rejected founders and product managers wondering why nobody will be their customer. It’s a sad place to be — I know from experience. Save yourself the hassle and explore the Discovery Delta instead.

The challenge with discussion guides isn’t usually that you can’t figure out the right questions to ask — it’s that you have too many questions to ask in the first place.

The blank page challenge in discovery research is the opposite of writer’s block. Rather than agonizing over an empty page all day, you’ve likely got 101 potential interview questions brainstormed in under 15 minutes. The challenge becomes keeping the list of questions short rather than sprawling and unmanageable.

But why is this a problem at all? Surely having a longer list of questions gives you a better chance of finding a golden nugget insight during your interview?

The root of this problem is a misunderstanding of your role as a discovery researcher. Your primary objective isn’t to ask great questions, it’s to listen for moments that surprise you and then pull on those loose threads. If you’re immediately planning your next question, then you’re doing it wrong.

Will the discussion guide be helpful once that loose thread has been identified? Nope! At that stage, you don’t need any fancy research methods — you just need to ask ‘Why?’ a couple of times.

But this still begs the question; what should my first question be in order to get them talking in rich detail about a specific memory? And what memory should I ask them to focus on?

This is the real challenge of discovery research — articulating your research objective.

If you’re a pre-product founder, your discovery research is likely focused on a specific occasion, use case or customer objective. You want to zoom in as much as possible on this singular activity with a homogenous sample of target customers to learn about what they’re trying to accomplish, why that’s important to them, and what’s getting in their way.

If you’re part of a product team doing discovery research, you’ll want to start by first understanding what company objective you’re trying to impact and then map out the opportunity areas (unmet needs, pains, desires) underneath that (ie. the OST model). Taking ‘increase activation rate’ as an example company objective → dig into the experience new sign-ups have with your product to identify the barriers limiting their success.

In either case, you’ll want to articulate the focus of your research and then let your interviewee spend +80% of the time just talking about their personal experience. Your job is to give them space to speak from their perspective, watch out for unexpected insights, and show some genuine curiosity.

There’s a reason why I’ve been pretty binary about discussion guides for discovery research throughout this essay. We’ve been missing one important point of nuance that needs to be explained…

Discovery interviews are completely different depending on whether you’re at the breadth or depth stage of the process. That’s where The Discovery Sandwich comes in. It’s a simple framework for splitting discovery research into three clear steps:

You don’t even need a discussion guide for the Breadth step of the Discovery Sandwich. Just ask them to talk about a specific recent time that they tried to accomplish the key activity you’re researching.

A discussion guide can be useful in the final stage of the Discovery Sandwich where you want to get really deep into the details and context that surrounds the specific pain point you’ve validated as highest priority. But the same logic applies — your job as a researcher is still to find and pull on loose threads that lead to unexpected insights, not to fill the interview with pre-planned questions.

If you’re at the start of a discovery research project, you’re better off answering these three questions instead of diving into discussion guide mode:

That’s everything. Time for you to go find some loose threads to pull :)

Daniel