The demo videos are compelling, and the pitch is always some version of the same thing: why spend time and money talking to real users when AI can simulate them for you?

We decided to find out if any of that holds up. Over the course of February and March, The Good’s team of UX researchers and strategists ran hands-on evaluations of six AI user research tools.

We tested them against real client projects, comparing outputs side-by-side with findings from our established methods, and sitting through demos with enough pointed questions to make the sales reps uncomfortable.

Our answer isn’t a simple thumbs up or thumbs down. Some of these tools are genuinely useful for the right team, in the right situation. Others are impressive on the surface, with not much underneath. And nearly all of them, once you get past the marketing language, will quietly acknowledge they can’t replace real user testing.

Here’s what we actually found.

First: “AI user research” is not one category

Before getting into the tools, it’s worth noting something that may not be intuitive to all. “AI user research” is a catch-all term that covers fundamentally different capabilities. Just as we have a variety of research tools and methods as an expert UX agency, the AI tool market includes a variety of tools with fundamentally different capabilities.

As we went deeper, we found most tools fall into one or more of these buckets:

• AI-assisted study setup: Helps you design a study or write a test plan.
• AI-moderated interviews: Replaces a human moderator with AI-guided conversation.
• Synthetic users: Generates AI personas that simulate user responses.
• AI follow-up questions: Dynamic probing within surveys or tests based on participant responses.
• AI analysis and synthesis: Themes survey responses, generates summaries, builds highlight reels.
• AI-driven roadmap and recommendation tools: Scans a site and generates prioritized UX recommendations.
• AI-powered heatmaps: Predicts visual attention without requiring real user data.

Knowing which category a tool belongs to matters because it changes what you should expect from it and what you shouldn’t.

The tools we evaluated

1. Synthetic Users

Category: Synthetic users/AI-assisted study setup

What it does:

Generates AI user profiles and simulates how those users would respond to a screenshot or Figma prototype, producing a full usability report with findings, quotes, and prioritized recommendations.

What we did:

One of our strategists ran the same test through Synthetic Users and through PlaybookUX with real recruited participants. It was the same landing page, with the same research questions. This is as close to a controlled comparison as we could get.

Where the findings matched:

• Both groups flagged information overload and cluttered design as major issues.
• Both raised privacy concerns about submitting a phone number.
• Both produced skepticism about the page’s bold marketing claims.
• Severity rankings were similar across both sets, and some of the language in the synthetic “user quotes” was remarkably close to what real participants said.

Where they diverged:

Synthetic Users can only test screenshots or Figma prototypes, not live URLs. No live interactions, form fills, or navigation behavior. In that way, real users provided behavioral data (where they actually clicked, how long they hesitated, what they scrolled past) that synthetic users can’t replicate.

The report itself output eight sections, including executive summary, task-by-task analysis, error patterns, user flows, learnability, satisfaction ratings, recommendations, and sub-sections. Each section largely repeated the same three or four findings in different formats. An experienced researcher would synthesize this into a focused, actionable deck while the tool generates volume.

The bottom line:

Synthetic Users got the big, surface-level findings right. If your question is “are there obvious issues with this page?”, it can answer that quickly. If your question is “how do real users actually interact with this experience?” it falls short. Think of it as a fast, automated heuristic review. It’s useful as a starting point, not a replacement for behavioral data.

2. Uxia

Category: Synthetic users/prototype testing

What it does:

Creates custom AI-generated users based on the audience information and test plan you provide (a step you can review and refine manually before the test runs). Those users then move through your prototype, and the tool automatically produces a synthesized report with ranked themes, findings, and a shareable output. No manual analysis required.

What we did:

One of our researchers gave Uxia’s team a Figma prototype of a site element that we had already tested and synthesized using Lyssna. This gave us a direct basis for comparison between their AI-generated output and our real user findings.

What worked:

The output is genuinely robust. Themes are already pulled and ranked, the report generates automatically, and it’s ready to share without anyone watching hours of recordings first. For an in-house team without a dedicated researcher, that’s a real time savings.

The AI users also flagged the same top finding that we identified in our test with real users. That kind of alignment on a specific, nuanced finding was notable.

Uxia positions itself honestly as a supplement to real user testing, not a replacement for it. They expect their users to be running studies with real participants alongside the tool, and they’re upfront about that. Researchers using their tool actually conduct more research because of the fast turnaround, not less.

Where they diverged:

AI users interpreted placeholder imagery as real content and confused a navigation menu for a standalone page.

Our team’s assessment: it doesn’t have the emotional intelligence a human user would.

What Uxia catches are surface-level friction points, including broken flows, confusing layouts, and missing content hierarchy. What it misses are the nuanced reactions that drive the most valuable optimization decisions.

The deeper limitation is scope. Many of the test types we run with human participants simply cannot be conducted with synthetic users. If 20 out of 30 real users say something similar, that’s a trustworthy signal built from independent behavior. If AI generates 30 synthetic responses that say the same thing, that’s one opinion multiplied.

Price is custom per team.

The bottom line:

Uxia works best as a pre-step. Running a prototype through it before live user testing to catch dead ends early, or to inform A/B test concepts, could be helpful. It’s not a replacement for behavioral research. The tool’s honest positioning about this was one of the more refreshing things we encountered in this evaluation.

3. Maze

Category: AI-moderated interviews / unmoderated testing

What it does:

Unmoderated usability testing platform that’s bolting on AI features, including AI-moderated interviews. Functionally similar to Lyssna, with AI moderation as the main differentiator.

What we did:

Our team ran a full walkthrough and tested its core capabilities.

What we found:

Maze predates AI. It’s a standard unmoderated testing tool that’s adding AI capabilities, not a purpose-built AI research solution. The AI moderation feature is built for teams that run a high volume of moderated studies and want to scale without adding headcount.

The AI follow-up question feature, which probes participants based on their responses, felt shallow in practice. It pulls a word from what someone typed and asks them to elaborate. One of our team members called it “advanced survey piping.” It’s an improvement over a static questionnaire, but it’s not a substitute for a skilled moderator who follows a line of inquiry.

The bottom line:

This is a capable unmoderated testing tool. The AI moderation pitch is most relevant to agencies or in-house teams running dozens of moderated sessions monthly. If you’re already using Lyssna and happy with it, there’s no compelling reason to switch.

4. Strella

Category: AI-moderated interviews/analysis and synthesis

What it does:

Replaces human moderators with AI-guided voice interviews, then auto-generates highlight reels, segmentation analysis, and synthesized findings reports.

What we did:

One of our researchers completed a demo and detailed review of capabilities and pricing.

What we found:

Strella’s synthesis features are genuinely interesting. Auto highlight reels, AI-generated segmentation, and an analysis interface that lets you ask questions of your data are all capabilities that could save significant time for teams running large volumes of qualitative research.

The problem is the price at $5,000 or more per project, not including participant recruitment or incentives. That math only works for organizations doing frequent, large-scale interview research.

We also acknowledge a gap in our evaluation here: we weren’t able to run a direct comparison of a real moderated interview against an AI-moderated one, because we don’t often conduct live moderated sessions for clients. Before making a definitive claim about quality differences, we’d want to test that directly. What we can say is that the tool solves problems a specific type of research operation has, not most in-house optimization teams.

The bottom line:

Potentially compelling for agencies or enterprise teams doing 20+ moderated studies a year. At current pricing, it’s a hard sell for most others. The synthesis capabilities are the most interesting part of the product, and we will be watching for those features to appear in more accessible tools.

5. Baymard UX-Ray

Category: AI-driven roadmap and recommendation tool

What it does:

Scans a website and generates a prioritized UX recommendation report, pulling from Baymard’s extensive research library to categorize and rank issues by page type and severity.

What we did:

We evaluated UX-Ray’s output against a real site, reviewed the tool’s methodology, and attended a Baymard-led NNG webinar where the founders discussed AI accuracy in UX recommendations.

What we found:

UX-Ray generated 342 UX insights for one site, a number that sounds impressive until you’re in the report and realize that quantity isn’t the same as usefulness. Many of the insights are gated behind paid tiers, and without the ability to prioritize by business impact, revenue potential, or implementation effort, a list of 342 findings is as overwhelming as it is informative.

The tool’s presentation is polished: dynamic, clickable, and organized by page type with thumbnail previews. And Baymard’s content library is a trusted source in UX research, whose credibility carries into the tool.

But the more fundamental limitation isn’t accuracy, it’s context. UX-Ray scans your site against a library of best practices and known UX patterns. It has no visibility into who your actual users are, how your specific audience behaves, or where your real conversion friction lives.

Entering a URL without that context assumes a lot. A recommendation that’s technically correct by best-practice standards may be irrelevant, or even counterproductive, for your particular visitors and traffic mix. Best practices are a starting point, not a strategy. That’s as true here as it is anywhere else in optimization.

Mid-tier pricing is $399 per month.

The bottom line:

Useful for teams that want a structured starting point for a UX audit and have the expertise to evaluate and filter the output. It’s not a replacement for a research-informed optimization strategy. The accuracy caveat matters; a list of 342 recommendations that’s 70–95% correct still requires an expert to separate the signal from the noise.

6. Brainsight

Category: AI-powered heatmaps

What it does:

Generates predictive attention heatmaps without requiring real user data, using AI trained on eye-tracking studies to model where users will look on a given page.

What we did: Unlike the other tools in this evaluation, we already use Brainsight in select client work. We’ve used it extensively enough to have a genuine, experience-based opinion.

What we found:

Of all the tools in this evaluation, Brainsight is the one we recommend most readily. But we present it with caveats, because that’s the honest way to use it.

The predictive heatmaps are reliable as a starting point. The tool reads contrast, copy, imagery, and dark areas on screen and makes assumptions about where human attention will land. That works often enough to be useful.

They also compare favorably to DIY AI heatmap alternatives (which our team found consistently unreliable), and the tool is priced accessibly enough to function as a genuine entry point for teams that haven’t yet invested in full heatmap research.

But it’s modeling visual salience, not actual user behavior. A true heatmap might show no heat on a long block of text that the AI flagged as a high-attention area, because real users navigated away without reading it. The AI doesn’t know that. It sees contrast; it doesn’t see intent.

So, this is a good starting point, not a definitive picture. If you want heatmap data you can trust completely, that comes from real users in a full engagement.

Here’s how we’d describe Brainsight to any client considering it: it gets you to 70% of the answer faster and cheaper than doing nothing. You’ll see where attention concentrates, where it drops off, and what’s fighting for visual priority.

The remaining 30%, understanding why users look where they look, what they do next, and what it means for your conversion strategy, is where a full optimization strategy makes the difference.’

Brainsight is also adding AI-generated recommendations following the heatmap output, a feature we haven’t fully evaluated yet. We’ll be watching it closely.

The bottom line:

This is a tool we use and would actively recommend as a starting point. Best positioned as an affordable entry into attention data, with the honest caveat that real engagement data tells you more.

What we learned across all of it

After evaluating all six tools, a few themes cut across the whole category.

They find the obvious. They miss the subtle.

In every comparison, AI-generated findings matched the surface-level issues an experienced researcher would spot in the first ten minutes of reviewing a page, for example, information overload, privacy friction, and confusing hierarchy.

The gap shows up in depth: navigation hesitation, emotional reactions, and the unexpected workaround a user invents that tells you your information architecture is broken. For high-stakes optimization decisions, the subtle findings are where the value lives.

More output is not better output.

Volume was the consistent way these tools tried to signal quality. 342 UX insights. Eight report sections for a single landing page. 12-page persona profiles in under a minute. Quantity without prioritization and context is noise. A skilled researcher delivers fewer, better, more actionable insights and knows which ones actually matter.

They’re genuinely useful for teams starting from zero.

This is worth saying clearly. An in-house optimization team that has never run a user test would benefit from these tools. Getting 70% of the answer is better than getting none.

These tools lower the barrier to research-informed decision-making. The risk isn’t using them, it’s treating their output as final rather than as a starting point that needs validation with real users.

The best use cases aren’t what the tools advertise.

The most promising applications we found weren’t the primary pitch of any tool we evaluated. Running a prototype through a synthetic user tool before live user testing to catch dead ends. Using Brainsight as a fast stakeholder-conversation starter. Using AI synthesis tools to surface patterns in data that a team has already collected but hasn’t had time to analyze. None of these tools market themselves this way, which our team found consistently surprising.

The vendors themselves will tell you.

This was the most telling finding of the entire evaluation. Every single tool vendor, once you moved past the landing page and into a real conversation, acknowledged that their tool won’t replace real user testing. When the sellers aren’t making the replacement claim, pay attention.

When to use AI research tools (and when not to).

AI tools earn their place when you’re tracking patterns over time, when the problem is well-defined and the stakes are low, when you need directional input quickly and the alternative is doing nothing, or when you’re QA-checking a prototype before investing in live user testing.

Keep humans in the lead when the decision is high-stakes, when you need behavioral data (not just stated responses), when you’re entering an unfamiliar market, or when you need findings you can defend with evidence.

For most teams, the answer isn’t either/or. These tools slot into a research process as a first step, a pre-launch check, or an accelerator for analysis you’re already doing. Their ceiling is lower than the marketing suggests, and their floor is higher than the skeptics give them credit for. Use them where they fit.

Frequently asked questions on AI user research

Can AI replace user research?

As of right now, no. And the vendors building these tools will tell you the same thing.

AI research tools can surface obvious usability issues, generate directional insights quickly, and lower the barrier to research-informed decision-making for teams that have never run a study before.

What they can’t do is replicate real user behavior: the navigation hesitation, the emotional response, the unexpected workaround that tells you something important about your experience.

For low-stakes, directional questions, AI tools are a reasonable starting point. For decisions that matter, real users are non-negotiable.

What is the difference between Synthetic Users and Uxia?

Both tools generate AI-simulated users to evaluate a design, but they serve slightly different purposes. Synthetic Users runs AI personas through screenshots or Figma prototypes and produces a full usability report with findings, quotes, and severity ratings, functioning most like an automated heuristic review.

Uxia takes a similar approach but focuses more specifically on prototype testing and positions itself explicitly as a first step alongside, not instead of, real user research.

In our side-by-side comparisons, both tools got the big surface-level findings right and missed the behavioral nuance. Uxia’s honest framing about its own limitations stood out as a green flag.

Is Brainsight accurate?

In our experience, yes. More so than DIY AI heatmap alternatives, which our team found consistently unreliable.

Brainsight’s predictive heatmaps are trained on real eye-tracking data and produce results we’ve found dependable enough to use in client sprint work. That said, predictive heatmaps model where users are likely to look based on visual patterns, they don’t capture actual user behavior, intent, or what users do after their attention lands somewhere.

We use Brainsight as a fast, accessible starting point. Real engagement data from actual sessions tells a more complete story.

How accurate are AI-generated UX recommendations?

According to Baymard’s own founders, AI-generated UX recommendations are generally around 70% accurate across the industry.

Baymard claims their UX-Ray tool performs at approximately 95%, but even at that rate, a meaningful portion of recommendations in any given report shouldn’t be implemented without validation.

The more important point: Baymard itself says all AI-generated recommendations require testing before you act on them. A tool that generates hundreds of insights you still need to verify manually isn’t saving as much time as the pitch suggests.

When should a team use AI user research tools?

AI research tools make the most sense when the alternative is doing no research at all, when you need quick directional input on a well-defined and lower-stakes question, when you’re doing pre-launch QA on a prototype before investing in live testing, or when you have existing data that needs faster synthesis.

They make the least sense when you’re making high-stakes optimization decisions, entering an unfamiliar market, or need findings you can defend with behavioral evidence. For those situations, real users and experienced researchers aren’t optional; they’re the whole point.

Do AI user research tools save time?

For teams with established research processes, the promised time savings largely didn’t materialize in our evaluation. Research teams can build test plans in their sleep and likely use AI to assist with analysis.

The tools that promised speed often delivered volume, lengthy reports that repeated the same findings across multiple sections, requiring a researcher to synthesize the synthesis.

For teams earlier in their research maturity, the time savings are more real: automating analysis and report generation genuinely helps when the alternative is doing it manually from scratch. But they will likely be bogged down in these unnecessarily long reports.

The verdict

We say the same thing about AI user research tools that we say about best practices: they’re a starting point, not a strategy. They get teams that have never done research to 70% of the answer. For a team with established processes and real users to test with, they don’t solve problems we have.

The hype runs well ahead of the utility, and the most dangerous outcome isn’t a team using these tools and getting incomplete results; it’s a team using them and thinking they have the full picture.

The last 30% of research quality, the part that connects real human behavior to your most important optimization decisions, still requires real users, real data, and experienced researchers who know what to do with both.

Not sure where AI tools fit in your research process…or if they should? Our team has done the testing. Book a call and let’s talk through it.

The post We Tested 6 AI Research Tools Against Real Users. Here’s What We Found. appeared first on The Good.

They don’t need research for research’s sake. They need a clear path from “we have a problem” to “here’s what to do about it.”

Research should systematically move you from understanding problems to implementing solutions. No guesswork. No dead ends. Just a repeatable process that gets you actionable answers.

That’s what a double diamond UX research framework does.

The double diamond UX research framework: design the right thing, then design the thing right

We wanted to share more about the double diamond UX research framework after seeing too many teams jump straight from problem to solution, without doing the work of understanding whether they’re solving the right problem in the first place.

The framework works because it forces teams to slow down in the right places and speed up in others. The framework is named for its diamond shape that emphasizes diverging to explore, then converging to decide.

You do this twice. First to ensure you’re solving the right problem, and then again to ensure you’re solving it the right way.

The first diamond is to design the right thing:

​​Expand your understanding of the problem space, then narrow it to the most important opportunities.

1. Define problems (research)
2. Deduce opportunities (synthesis)

The second diamond is to design the thing right:

Explore multiple solution approaches, then validate which one actually works.

1. Create choices (ideation)
2. Evaluate options (implementation)

Each phase has a specific purpose, uses different research methods, and answers different questions.

When to use this framework

This UX research framework works best when:

• You’re stuck. Something’s not working, and you’re not sure why. The framework helps you diagnose root causes instead of treating symptoms.
• You have options. You’ve got a few directions you could pursue, but need validation before committing resources.
• Stakes are high. You’re making changes to core experiences, entering new markets, or redesigning critical flows where getting it wrong is expensive.
• You want speed without guesswork. You need answers quickly but can’t afford to be wrong.
• You need team alignment. Different stakeholders have different opinions, and you need objective data to move forward.

How it works phase by phase

But of course, this framework is only useful if you know how to apply it! Here’s a breakdown of each phase: when you need it, which research methods to use, and real examples of how we’ve used it to solve specific problems.

Phases 1-2: Design the right thing (generative research)

The first diamond uses the same research toolkit for both defining problems and deducing opportunities. What changes is how you apply these methods:

Phase 1 (Define Problems): You’re gathering data, uncovering what’s broken, and understanding the full scope of user pain points.

Phase 2 (Deduce Opportunities): You’re synthesizing that data, identifying patterns, and prioritizing which problems matter most.

Research toolkit:

• User interviews: Understand motivations, mental models, and pain points
• User testing: Watch real users interact with your product to identify friction
• Heatmap analysis: See where users actually click, scroll, and spend time
• Data analysis: Identify patterns in behavior and correlations with outcomes
• Journey analysis: Map complete user experiences to find friction points
• Surveys: Gather quantitative data on behaviors and preferences
• Ethnographic analysis: Observe users in their natural context to understand real-world behavior

Phase 1: Define problems through generative research

Before you can solve anything, you need to understand what’s actually broken. This is where generative research comes in. This research is designed to uncover opportunities and pain points you might not even know exist.

When you need this phase:

• You know something’s not working, but can’t pinpoint what
• You’re entering a new market or launching a new product
• Conversion rates have plateaued, and you’re not sure why
• You’re getting conflicting feedback from different user segments

Case study:

The problem: A SaaS productivity platform noticed that users who adopted their collaboration feature had 40% better retention, but only 12% of new users ever tried it. The product team had already attempted several fixes, including in-app tooltips, email campaigns, and adding the feature to the main navigation, but none moved the adoption needle. They knew they had a problem, but couldn’t pinpoint the root cause.

What we did: We ran user testing with 20 recent sign-ups, watching them complete typical workflows while thinking aloud. We analyzed heatmap data to see where users looked during key moments. We conducted interviews with 12 users (6 who adopted the feature, 6 who didn’t) to understand their mental models and pain points. Finally, we analyzed behavioral data across 50,000 user sessions to identify patterns in how people discovered features.

What we found: Most users saw the feature prompts, so the issue wasn’t awareness. The problem was timing and relevance. Users ignored prompts for the collaboration feature when they appeared during solo work. But when users received documents with stakeholder comments, they actively looked for ways to coordinate feedback. The feature solved a real problem, but only at specific moments. Showing it at the wrong time made it feel like noise.

Phase 2: Deduce opportunities through synthesis

Raw research data doesn’t solve problems, but synthesis does. This phase is about taking everything you learned and distilling it into clear, prioritized opportunities.

When you need this:

• You have research findings, but need to prioritize
• Stakeholders are divided on which direction to pursue
• You need to build a business case for investment
• You’re trying to align multiple teams around a shared strategy

Case study:

What we did: We took the same data from Phase 1 but shifted our analysis to look for patterns and opportunities. We mapped user journeys to identify the specific trigger moments when collaboration pain became acute. We segmented users by workflow type to understand who encountered collaboration needs most frequently. We analyzed the correlation between feature discovery timing and adoption rates.

The opportunity: Instead of promoting the feature broadly or adding more persistent navigation, we identified a single high-value trigger: show the collaboration feature contextually when users open documents containing comments or tracked changes. This represented the precise moment when users experienced the pain this feature solved.

The impact potential: Based on session data, 43% of new users encountered stakeholder-reviewed documents within their first week. If we could convert just 50% of those moments into feature trials, we’d nearly triple overall adoption.

What happened next: We had a clear, data-backed opportunity. Now we needed to figure out how to present this feature at that contextual moment. That’s where Phase 3 came in.

Phase 3: Create choices through competitive analysis

Once you know what problems to solve, you need to explore how to solve them. This phase is about generating multiple possible solutions and understanding how others have approached similar challenges.

Research toolkit:

• Competitive analysis: Study how industry leaders solve similar problems
• Landscape analysis: Identify patterns across different approaches

When you need this:

• You need to narrow down design directions
• Stakeholders want to see options before committing
• You’re redesigning an existing feature and need alternatives
• You’re unsure which approach will resonate with users

Case study:

The problem: We knew when to show the collaboration feature, but we didn’t know how. Should it be a modal that interrupts the workflow? A subtle banner? A tutorial overlay? Each approach had trade-offs, and stakeholders had different opinions about what would work.

What we did: We conducted a competitive analysis of 12 productivity and collaboration tools, studying how they introduced features at contextual moments. We analyzed landscape patterns across project management, document editing, and communication platforms to understand what users had learned to expect.

What we found: The highest-performing patterns shared three characteristics:

1. They acknowledged the user’s current task (“We see you’re reviewing comments”)
2. They offered immediate value related to that task (“Collect all feedback in one place”)
3. They provided a low-friction way to try (“Add your first comment now”)

The choices we created: Based on these patterns, we designed three variations:

1. Inline prompt: A compact banner within the document that appeared next to existing comments
2. Modal introduction: A full-screen overlay explaining the feature with a demo
3. Progressive disclosure: A subtle sidebar element that expanded when users hovered near comments

Where this led: We had three viable approaches grounded in what worked for others. In Phase 4, we tested which resonated most with actual users.

Phase 4: Evaluate options

Before you ship, you need to validate. This final phase uses evaluative research methods to test solutions quickly and make confident decisions.

Research toolkit:

• User testing: Validate whether users can actually complete tasks
• Prototype testing: Test concepts before full development
• Rapid testing: Get fast feedback on specific design decisions (preference tests, first-click tests, tree tests, design surveys, card sorts)
• Sentiment testing: Measure emotional response to different approaches
• A/B testing: Compare variations with real users in production

When you need this:

• You have a few strong directions and need to pick one
• You want to validate before investing in full development
• You need fast feedback to maintain momentum
• You’re making changes to high-traffic, high-value experiences

Case study:

The problem: All three approaches looked good on paper, but we needed to know which one users would actually respond to. We didn’t have time to build and A/B test all three variations in production. We needed a faster way to validate the direction before investing in full development.

What we did: We created rapid prototypes of all three approaches and ran user testing with 24 people who matched our target audience. We showed each user a realistic scenario: opening a document with stakeholder comments. Then we measured task completion (could they figure out how to use the collaboration feature?), time to first action (how quickly did they engage?), and sentiment (how did they feel about the interruption?).

What we found: The inline prompt won on every metric, and users described the inline prompt as “helpful” and “right when I needed it,” while the modal felt “intrusive” and progressive disclosure was “too easy to miss.”

The decision: The team built and shipped the inline prompt, and feature adoption increased.

Why this worked: By following all four phases of the double diamond ux research framework, we didn’t just guess at a solution. We systematically moved from “we have a problem” (low adoption) to “here’s the root cause” (timing and relevance) to “here’s how others solve this” (contextual introduction patterns) to “here’s proof this will work” (validated direction through testing). Each phase built on the last, reducing risk at every step.

Why this UX research framework works

The double diamond framework prevents the most common research mistakes:

It prevents solution jumping

Teams often skip straight from “we have a problem” to “here’s what we’ll build.” Our framework forces you to first understand the problem deeply (Phase 1), then synthesize opportunities (Phase 2) before exploring solutions (Phase 3) and validating them (Phase 4).

It matches research methods to questions

Generative research (Phases 1-2) answers “what problems exist?” Evaluative research (Phases 3-4) answers “does this solution work?” Using the wrong method for your question wastes time and money.

It creates natural checkpoints

Each phase ends with a clear deliverable: defined problems, prioritized opportunities, multiple solutions, or validated direction. Stakeholders can weigh in at the right moments without derailing progress.

It scales to any timeline

Need an answer this week? Run a rapid test. Have a month? Go deeper with interviews and competitive analysis. Have a quarter? Complete the full cycle. The framework adapts to your constraints while maintaining rigor.

How you can adapt the framework to increase revenue from your digital experience

While it would be ideal to run the full framework on all product decisions, we don’t always have the luxury of time. Here’s how we typically adapt the framework depending on project needs.

Fast decision (1-2 weeks): Skip straight to Phase 4 with rapid testing when you have clear options and need directional data quickly.

Optimization project (2-4 weeks): Focus on Phases 1-2 (define problems, deduce opportunities) when you need to diagnose what’s broken and prioritize fixes.

Strategic initiative (4-8 weeks): Full cycle through all four phases when you’re entering new territory or making high-stakes changes.

Ongoing partnership: Rotate through phases as needs evolve. We may use competitive analysis this month, user testing next month, and rapid testing the month after.

The key is matching research depth to risk and constraints.

You’re not trying to run the most comprehensive study possible. Instead, you’re trying to give yourself the confidence to make the right decision with the time and budget you actually have.

Ready to solve your research challenge?

The double diamond framework gives you a proven way to move from problem to solution. Whether you need a quick rapid test to validate a design decision or a comprehensive research engagement to inform strategy, we adapt this framework to your specific needs.

The teams we work with don’t need more data; they need clarity. They need to know what’s broken, why it matters, and what to do about it. That’s exactly what this UX research framework delivers.

And when you work with The Good, here’s what research deliverables look like:

• Problem definition: Clear documentation of user pain points, friction in the experience, and opportunities ranked by impact and effort.
• Competitive analysis: Side-by-side comparison of how others solve similar problems, with specific recommendations for what to adopt, adapt, or avoid.
• User testing results: Video clips of real users, annotated with insights, organized by finding severity, and accompanied by specific design recommendations.
• Rapid test reports: Statistical analysis of user preferences with clear winners, qualitative feedback explaining why users chose what they did, and next-step recommendations.
• Strategic recommendations: Prioritized roadmap of what to build, test, or optimize based on research findings and business goals.

Everything is actionable. We don’t just tell you what we found, we tell you what to do about it.

Want to see how this framework would apply to your specific challenge? Let’s talk about your research needs.

The post How to Use the Double Diamond UX Research Framework to Solve Real Problems appeared first on The Good.

It’s tempting. The allure of real-time, live data showing exactly which price point converts better feels like the holy grail of product optimization. Fire up your testing platform, split traffic between $29 and $39, and let the numbers tell you what to charge.

But price testing is an ethical and legal minefield that can damage customer trust and put your brand at risk.

After 16 years of optimizing digital experiences, we’ve seen this scenario play out dozens of times. A client comes to us excited about testing prices, we dig into what that actually entails, and we end up recommending something entirely different: pricing research.

The difference matters. A lot.

Why we don’t recommend traditional price testing

While A/B price testing isn’t explicitly illegal in most jurisdictions, it occupies a murky grey area that should make any brand leader pause.

The legal landscape

In the United States, price testing is generally legal. The Robinson-Patman Act prohibits certain forms of price discrimination, but its scope is narrow, primarily applying to business-to-business sales of commodities where different pricing harms competition. For most consumer-facing businesses, the Act rarely applies, and violations are difficult to prove.

In the European Union, however, the situation is different. According to EU law, charging customers differently based solely on their nationality is illegal. Even in random A/B tests where nationality isn’t the determining factor, if a French customer pays more than a Belgian customer for the same product, you could face fines if complaints are filed with the European Consumer Centre.

Recent research published in the Journal of Revenue and Pricing Management highlights how pricing executives must now navigate the triangulation of legal constraints, ethical considerations, and algorithmic decision-making when setting prices.

The consumer perception problem

Beyond legality, there’s the court of public opinion.

A 2022 study from Phiture found that different generational groups react very differently to personalized pricing. While Gen X consumers sometimes try to “game the system” (clearing cookies or using incognito mode to search for better deals), many Millennials and most Boomers react negatively when they discover they’re being charged different prices than other customers.

The Instacart case proves this isn’t theoretical. A Consumer Reports survey conducted in September 2025 found that 72% of Instacart users did not want the company to charge different prices to different users for any reason.

When the investigation revealed the extent of the price testing, customers described feeling “manipulated,” “deceived,” and said they were “not as trusting of a company that practices that.” One volunteer specifically said: “All prices should be the same for everybody, whether you’re rich or poor… some people are going to have to fight back against that system.”

Within weeks of the investigation’s publication, Instacart discontinued the practice entirely, a clear signal that the reputational risk outweighed any revenue optimization gains.

Most consumers view price discrimination as fundamentally unfair, even when it’s legal. When customers discover they paid more than someone else for the exact same product at the exact same time, trust erodes quickly. And once lost, that trust is expensive to rebuild.

The technical limitations

While platforms like Shopify support native price testing functionality, testing tools typically don’t have the infrastructure to modify your actual pricing across different customer segments reliably. Even if you’re comfortable with the ethical considerations, there are practical barriers.

The margin problem

As expert pricing research from Paddle notes, even legal pricing strategies become unethical when they ignore fundamental business health. Simply optimizing for conversion without understanding contribution margin can lead you to “win” tests that actually hurt your bottom line.

Sure, we might see that Product A sold more units than Product B at a given price point, but which product has better margins? That difference fundamentally impacts whether a price change is actually driving profitability or just revenue.

The smarter alternative: pricing research

After explaining these challenges, clients often ask: “So what should we do instead?”

Structured pricing research. Rather than testing prices live on your site where you’re charging real customers different amounts, conduct research that reveals willingness to pay, price sensitivity, and optimal price points before you go to market.

Pricing research gives you the insights of price testing without the ethical baggage, legal risk, or customer trust issues. Here are the primary methodologies we recommend:

Van Westendorp Price Sensitivity Meter

Developed by Dutch economist Peter Van Westendorp in 1976, the Price Sensitivity Meter (PSM) remains one of the most effective ways to identify acceptable price ranges for products.

The methodology is elegant in its simplicity. You survey your target customers with four key questions:

1. At what price would you consider this product too expensive to purchase?
2. At what price would you consider this product expensive, but still worth considering?
3. At what price would you consider this product a bargain?
4. At what price would you consider this product so inexpensive that you’d question its quality?

By plotting cumulative responses to these questions, you can identify several critical price points:

• Point of Marginal Cheapness (PMC): The intersection of “too cheap” and “expensive” lines, which indicates your lower bound
• Point of Marginal Expensiveness (PME): The intersection of “too expensive” and “cheap” lines which indicates your upper bound
• Optimal Price Point (OPP): Where an equal number of respondents describe the price as exceeding either their upper or lower limits
• Indifference Price Point: Where the same number of people think the price is “too expensive” as those who think it’s a “bargain”

According to research from SurveyKing, Van Westendorp is particularly valuable for identifying pricing thresholds and overall market perceptions without putting actual customers in a position where they’re being charged inconsistently.

When to use it: Van Westendorp excels for new-to-world products where you’re establishing an initial price point, or when repositioning an established product in a new market segment. It’s also fast to implement because you can run a Van Westendorp study in days, not weeks.

Limitations to know: The method focuses solely on price perception without considering product features or competitive context. It also can’t predict actual purchase behavior, only price expectations. As noted in research from Conjointly, if your product has multiple configurations or you need to understand feature-specific value, other methods may be more appropriate.

Conjoint analysis

If Van Westendorp is the quick mission, conjoint analysis is the full strategic assessment.

Conjoint analysis reveals how customers value different product attributes, including price, by forcing them to make trade-offs between product profiles. Rather than asking “What would you pay for this?”, conjoint presents respondents with complete product profiles that vary across multiple dimensions (features, brand, price, etc.) and asks them to choose which they’d buy.

For example, a project management software might test profiles varying:

• Number of team members included (5, 15, or 50)
• Storage capacity (10GB, 50GB, or 250GB)
• Integration options (3, 10, or unlimited)
• Price ($19/month, $49/month, or $99/month)

Respondents see sets of 3-4 profiles at a time and select their preference. The pattern of choices reveals the relative value of each attribute, including price sensitivity.

Choice-based conjoint (CBC) is particularly powerful for pricing research because it simulates realistic purchase scenarios. Respondents don’t know you’re primarily interested in pricing; they’re just choosing products they’d actually buy. This approach delivers more honest insights than directly asking about willingness to pay.

Why it works: Conjoint lets you measure price elasticity by brand, understand optimal feature-price combinations, and run market simulations to predict revenue and share. Research from GLG shows that with conjoint data, you can model hypothetical scenarios: “If we add this feature and increase the price by $10, how many customers will we gain or lose?”

When to use it: Conjoint shines when you need to understand how price interacts with product features, or when you’re pricing complex offerings with multiple tiers or bundles. It’s the gold standard for SaaS pricing strategy because it captures the reality that customers evaluate price in context, not isolation.

What to expect: Conjoint requires more upfront investment than Van Westendorp, both in study design and sample size. You’ll need larger respondent pools (typically 300+ for reliable results), and the analysis is more sophisticated. But the insights are proportionally richer.

Segmentation and historical data analysis

Sometimes the best pricing insights are hiding in your own data.

Before running any new research, we always recommend examining your existing customer base through a segmentation lens. Different customer segments often have dramatically different price sensitivity.

Research from TRC Insights shows that price elasticity, the measure of how demand changes with price, varies significantly across customer segments. Enterprise buyers might be relatively price-insensitive (inelastic demand) for mission-critical tools, while small businesses might be highly price-sensitive (elastic demand) for the same product.

By analyzing your historical data, you can identify:

• Which segments have the highest lifetime value at different price points
• How acquisition cost varies by price tier across segments
• Retention patterns that indicate whether pricing is aligned with value delivery
• Upgrade and downgrade patterns that reveal price ceiling and floor effects

One telecommunications company we know of analyzed years of customer data to understand price elasticity by segment. They discovered that their “small business” segment was actually three distinct sub-segments with wildly different price sensitivities:

• one that behaved like enterprise (low elasticity)
• one that behaved like consumers (high elasticity)
• and one in between

This insight led them to redesign their entire pricing strategy with separate offers for each sub-segment, ultimately increasing revenue by 10%+.

When to use it: Always. Historical data analysis should be your starting point for any pricing decision. It’s low-cost (you already have the data), fast, and often reveals surprising patterns.

Gabor-Granger method

For a more direct approach to estimating demand curves, the Gabor-Granger method offers a middle ground between Van Westendorp and conjoint analysis.

The process is straightforward: show respondents a product at a specific price and ask if they’d buy it. If yes, show a higher price. If no, show a lower price. Continue until you map out their individual purchase threshold.

Aggregate these responses across your sample, and you can build demand curves that predict:

• The percentage of your market that will buy at each price point
• The revenue-maximizing price
• The volume-maximizing price
• Price elasticity at different levels

This can be particularly useful when you need quick market assessments and want to focus specifically on price sensitivity without evaluating multiple product attributes simultaneously.

When to use it: Gabor-Granger works well for single products or when product attributes are already determined, and you need to optimize pricing specifically. It’s faster than full conjoint but more direct about pricing than Van Westendorp.

Understanding price elasticity for better decisions

All of these methodologies ultimately help you understand price elasticity, how changes in price affect demand for your product.

Price elasticity is typically expressed as: % change in quantity demanded ÷ % change in price

Products with elastic demand (elasticity > 1) see large changes in demand with small price changes. Think luxury goods, or products with many substitutes. Products with inelastic demand (elasticity < 1) see relatively stable demand despite price changes. Think of necessities or products without good alternatives.

Understanding your product’s elasticity is crucial because it determines your pricing strategy’s impact. For elastic products, lowering prices can increase total revenue. For inelastic products, you might be leaving money on the table by not charging more.

Here’s what makes elasticity even more interesting: it’s not fixed. The same product can exhibit different elasticity depending on:

• Customer segment: Enterprise buyers vs. SMBs vs. individual consumers
• Time period: Demand becomes more elastic over time as customers adjust their behavior
• Market conditions: Economic downturns increase price sensitivity even for traditionally inelastic goods
• Price range: Products can be inelastic at low prices but highly elastic at high prices

Understanding these nuances helps you make smarter pricing decisions across your entire customer base, not just at a single price point.

A real example: how we approach pricing strategy

Here’s how these methodologies come together in practice.

A B2B SaaS company approached us, concerned that their pricing wasn’t optimized. They had three tiers ($49/month, $149/month, and $499/month) that had been set somewhat arbitrarily three years ago based on “what felt right” and competitive benchmarking.

Rather than jumping into A/B testing prices, here’s the path we recommended:

Phase 1: Data analysis

We started by analyzing their existing customer data:

• Segmented customers by industry, company size, and usage patterns
• Calculated lifetime value and retention by segment and tier
• Mapped upgrade/downgrade patterns to understand price ceiling effects
• Identified which features correlated with willingness to pay premium prices

This revealed that their “mid-market” segment was actually two distinct groups with different needs and willingness to pay.

Phase 2: Van Westendorp study

We ran a Van Westendorp survey with 400 prospects and recent customers across identified segments. This quickly established:

• Their $49 tier was perceived as “too cheap” by 30% of respondents, potentially signaling quality concerns
• There was an acceptable price range between $79-199 for their middle tier
• Their top tier had room to increase to $599-699 based on value perception

Phase 3: Conjoint analysis

With price ranges identified, we ran a choice-based conjoint to understand:

• Which features justified premium pricing
• How different customer segments valued different feature bundles
• Optimal price points for proposed new tiers

The conjoint revealed that their original three-tier structure was actually constraining revenue. There was demand for a fourth tier at $799/month for enterprise features, and their middle tier could be split into two offerings at $99 and $199.

Results

The company implements a new four-tier pricing structure ($69, $119, $239, $799) based on the research. Average revenue per customer would increase 23%. Customer acquisition would actually improve (lower entry price brings in more customers who later upgrade). Retention holds steady despite price increases because the value alignment was better

This approach would take 8 weeks, and the cost would be well worth it compared to the potential brand damage of customers discovering they’d been charged different prices in an A/B test, or the opportunity cost of not optimizing pricing at all.

Making the case for pricing research internally

If you’re reading this thinking, “this makes sense, but my team really wants to just A/B test prices,” here’s how to make the case:

Frame it as risk management

Price testing puts your brand reputation at risk. Pricing research gives you comparable insights without exposing you to customer backlash, legal concerns, or PR problems. Maintaining customer trust through transparent, ethical pricing practices is crucial for long-term profitability.

Emphasize the quality of insights

A/B tests tell you which price performed better in one specific context at one specific time. Pricing research tells you why that price works, how different segments perceive value, and how pricing interacts with features and positioning. Those insights compound over time.

Talk about margin, not just conversion

This one resonates with CFOs. Pure price tests optimize for conversion or revenue, but they don’t account for margin variation across products or customer acquisition costs across segments. Pricing research can be designed to optimize for profit, not just revenue.

Point to the technical limitations.

Most A/B testing platforms can’t reliably execute pure price tests anyway. You’d need to implement complex technical workarounds that introduce their own risks. Pricing research is straightforward to implement with existing survey tools.

Pricing strategy

The urge to price test is understandable. You want data-driven pricing decisions. You want to optimize this critical lever for growth.

But the best data doesn’t come from exposing real customers to different prices in an A/B test. It comes from structured research that reveals customer psychology, value perception, and willingness to pay without the ethical complications.

What you can do is test discount codes or promotional messaging. For example, we ran a test for a client where some visitors saw “$50 free shipping minimum,” while others saw “$75 free shipping minimum” or “$100 free shipping minimum.” In reality, everyone had a $50 minimum on the backend, but the messaging encouraged different customer segments to add more to their carts. This isn’t pure price testing; it’s messaging optimization that influences average order value.

We’ve spent 16 years helping ecommerce and SaaS companies optimize their digital experiences. When it comes to pricing, though, the most successful companies skip the shortcuts and invest in research that protects customer trust while delivering the insights they need.

The next time someone proposes A/B testing prices, ask them if they’ve considered the alternatives. The answer might surprise them and save your brand from an expensive mistake.

Let’s talk about your pricing strategy.

The post How to Test Your Pricing Strategy Without the Ethical Minefield appeared first on The Good.

The company had assembled a list of proof points they could showcase on their homepage: years in business, number of integrations, customer counts, implementation guarantees, security certifications, industry awards, analyst recognition, and more. But they only had space to highlight four of these benefits prominently below their hero section. They faced the classic messaging dilemma: which trust signals would actually move the needle with prospects evaluating B2B software?

This is where MaxDiff analysis becomes valuable. Instead of relying on stakeholder opinions or generic best practices, we could let their target buyers vote with data on what mattered most.

What makes MaxDiff analysis different from other survey methods

MaxDiff analysis (short for Maximum Difference Scaling) is a research methodology that forces trade-offs. Rather than asking people to rate items individually on a scale, MaxDiff presents sets of options and asks participants to identify the most and least important items in each set. This forced-choice format reveals true preferences because people can’t rate everything as “very important.”

Here’s why this matters: traditional rating scales often produce compressed results where everything scores high. When you ask customers, “How important is X on a scale of 1-10?” most people will hover around 7 or 8 for anything remotely relevant. You end up with a spreadsheet full of similar numbers and no clear direction.

MaxDiff cuts through that noise. By repeatedly asking “which of these five options matters most to you, and which matters least?” across different combinations, you build a statistical picture of relative importance. The math behind MaxDiff generates a best-worst score for each item, showing not just which options are preferred, but by how much.

For digital experience optimization, this methodology is particularly useful when you need to prioritize limited real estate on a website, determine which features to build first, or figure out which messaging will actually differentiate your brand.

How we structured the MaxDiff study for maximum insight

In the project for our client, we started by defining the target audience precisely. The company was a B2B SaaS platform serving mid-market operations teams, so we recruited 60 participants who matched their customer profile: director-level or above at companies with 50-500 employees, working in operations or supply chain roles, currently using at least two SaaS tools in their workflow, and actively evaluating solutions within the past six months.

From the initial audit and stakeholder interviews, we identified 11 potential trust signals the company could emphasize on its homepage. These included things like:

• Concrete numbers (customer counts, uptime percentages, integrations available)
• Credentials (security certifications, enterprise clients)
• Promises (implementation timelines, support response times, money-back guarantees)
• And more

Each represented something the company could truthfully claim, but we needed to know which ones would build the most trust with prospects evaluating the platform.

The survey design was straightforward. Each participant saw these 11 benefits randomized into multiple sets of five items. For each set, they selected the most important factor and the least important factor when considering whether to adopt this type of software. Participants completed several rounds of these comparisons, seeing different combinations each time.

This approach gave us enough data points to calculate a robust best-worst score for each benefit: the number of times it was selected as “most important” minus the number of times it was selected as “least important.” Positive scores indicate a strong preference, negative scores indicate a low importance, and the magnitude of the scores shows the strength of feeling.

The results revealed a clear hierarchy of trust signals

When we analyzed the MaxDiff results, the pattern was striking. The top-scoring benefits shared a common theme: they provided concrete evidence of proven reliability and satisfied users. The bottom-scoring benefits? They emphasized company scale and marketing visibility.

The four highest-scoring trust signals were clear winners. G2 or Capterra ratings scored 38 points (the highest possible), indicating this was nearly universal in its importance. The number of active customers scored 30 points. An implementation guarantee (“live in 30 days or your money back”) scored 25 points. And SOC 2 Type II certification scored 16 points.

These weren’t arbitrary marketing metrics. They were the specific signals that would make someone think, “this platform delivers real value and other companies trust them.”

The middle tier included operational details that registered as minor positives but weren’t decisive: the number of successful implementations (7 points), availability of 24/7 support (6 points). These signals suggested competence but didn’t particularly move the needle on trust.

Then came the surprises. Years in business scored -5 points, indicating it was slightly more often selected as “least important” than “most important.” The number of integrations available scored -11 points. AI-powered features claimed scored -15 points. Employee headcount scored -36 points. And recognition as a Gartner Cool Vendor scored -55 points, the lowest possible score.

Think about what prospects were telling us: “I don’t care that you have 200 employees or that Gartner mentioned you. Show me that real companies like mine trust you and that you’ll actually deliver on your promises.”

Why customers rejected company-focused metrics

The findings revealed an insight into trust-building that extends beyond this single company. B2B buyers weigh social proof and reliability guarantees far more heavily than they weigh indicators of company scale or industry recognition.

When a business talks about its employee headcount or analyst mentions, prospects interpret this as the company talking about itself. These metrics answer the question “How big is your business?” but not “Will this solve my problem?” From the buyer’s perspective, a larger team or Gartner mention doesn’t necessarily correlate with better software or smoother implementation.

By contrast, user reviews and customer counts answer the implicit question every prospect has: “Did this work for companies like mine?” A guarantee directly addresses risk: “What happens if implementation fails?” Security certifications address legitimacy: “Is this platform secure enough for our data?”

The AI-powered features claim scored poorly, likely because it felt trendy rather than practical. Prospects for this specific business weren’t primarily concerned about cutting-edge technology; they wanted a platform that would reliably solve their workflow problems. Leading with an AI angle, while possibly true, didn’t address the core decision-making criteria.

Years in business scored negatively for similar reasons. While longevity can signal stability, in this context, it didn’t address the prospect’s immediate concerns about implementation speed and user adoption. A company could be around for years while providing clunky software with poor support.

From insight to implementation: turning research into revenue

The MaxDiff analysis gave the company a clear action plan. We recommended implementing a four-part trust signal section directly below their homepage hero, featuring the top four scoring benefits in order of importance.

This meant reworking their existing homepage structure. Previously, they had emphasized their implementation guarantee in the hero area while burying customer counts and ratings further down the page. The research showed this approach had it backward. Prospects needed to see evidence of customer satisfaction first, then the implementation guarantee as additional reassurance.

We also recommended removing or de-emphasizing several elements they had been proud of. The employee headcount mention, the Gartner recognition, and several other low-scoring items were either removed entirely or moved to less prominent positions on the site. The goal was to prevent low-value signals from crowding out high-value ones.

The broader lesson here extends beyond this single homepage optimization. The MaxDiff results provided a messaging hierarchy that the company could apply across its entire go-to-market strategy. Email campaigns, landing pages, sales conversations, demo decks, and even their LinkedIn company page could now emphasize the trust signals that actually mattered to prospects.

When MaxDiff analysis makes sense for your business

MaxDiff is particularly valuable when you’re facing a prioritization problem with limited data. It works best in these scenarios:

• You have more options than you can implement. Whether that’s features to build, benefits to highlight, or messages to test, MaxDiff helps you choose wisely when you can’t do everything at once.
• Stakeholder opinions are conflicting. When internal debates about priorities can’t be resolved through argument, customer data settles the question. MaxDiff provides quantitative evidence for decision-making.
• You need to differentiate in a crowded market. If competitors are all saying similar things, MaxDiff reveals which specific claims will break through. Often, the winning messages are ones companies overlook because they seem “obvious” or “not unique enough.”
• You’re optimizing for a specific audience segment. Generic research about “customers in general” often produces generic insights. MaxDiff works best when you recruit participants who precisely match your target customer profile.

The methodology has limitations worth noting. It requires careful setup, and you need to know which options to test before you start.

If you don’t include the right benefits in your initial list, you won’t discover them through MaxDiff.

It also works best with a reasonably sized set of options (typically 5-15 items).

And the results tell you about relative importance, not absolute importance; everything could theoretically matter, but MaxDiff reveals the hierarchy.

How to use MaxDiff findings in your optimization strategy

Once you have MaxDiff results, the application extends beyond simply reordering homepage elements. The insights should inform your entire digital experience.

Your messaging architecture should reflect the importance hierarchy. High-scoring benefits deserve prominent placement, repetition across pages, and detailed explanation. Low-scoring benefits can either be removed or repositioned as supporting rather than leading messages.

Your testing roadmap should prioritize changes based on MaxDiff findings. If customer reviews scored highest in your study, test different ways of showcasing reviews before you test other elements. Let the data guide your experimentation priorities.

Your content strategy should emphasize what customers care about. If service guarantees scored highly, create content that explains the guarantee in detail, shares stories of when it was honored, and addresses common concerns. Build your editorial calendar around the topics MaxDiff revealed as important.

Your sales enablement should align with customer priorities. If the research showed that prospects value licensing credentials, make sure your sales team knows to emphasize this early in conversations. Create collateral that highlights the trust signals that matter most.

The most effective companies use MaxDiff as one tool in a broader research program. They combine it with qualitative research to understand why certain benefits matter, behavioral analytics to see how users interact with different messages, and continuous testing to validate that the predicted preferences translate into actual conversion improvements.

Turning guesswork into growth

The SaaS company we worked with started with a dozen possible messages and no clear sense of which would build trust most effectively with B2B buyers. After the MaxDiff analysis, they had a data-backed hierarchy that let them confidently restructure their homepage and broader messaging strategy.

This is the power of asking prospects the right questions in the right way. Not “do you like this?” which produces inflated scores for everything. Not “rank these 11 items,” which overwhelms participants and produces unreliable data. But rather, through repeated forced choices, revealing the true importance of each element.

If you’re struggling with similar prioritization challenges (too many options, limited space, stakeholder disagreement about what matters), MaxDiff analysis might be the tool that breaks through the noise. It transforms subjective opinion into statistical evidence, letting your prospects vote on what will actually convince them to choose your platform.

Ready to discover which messages actually resonate with your customers? The Good’s Digital Experience Optimization Program includes research methodologies like MaxDiff analysis to help you prioritize changes based on real customer preferences, not guesswork.

The post MaxDiff Analysis: A Case Study On How to Identify Which Benefits Actually Build Customer Trust appeared first on The Good.

The default answer has been A/B testing. But while A/B testing remains the gold standard for high-stakes decisions, it’s not always the right tool for every design change. Many teams have fallen into the trap of either testing everything (creating bottlenecks and slowing innovation) or testing nothing (making changes based purely on intuition).

There’s a better way. By understanding when different validation* methods are most appropriate, SaaS teams can make faster, more confident design decisions while maintaining the rigor needed for their most critical changes.

*Note: We know validation is a bad word in the research community because it implies “proving you’re right,” but we feel it’s easier to read and more quickly comprehensible for those not in research disciplines. We’re using “validation” in this article, but “evaluation” or “confirm or disconfirm” would be more acute in other settings.

The real cost of a bad experimentation strategy

When teams lack a clear strategy for validating decisions, they create what researcher Jared Spool calls “Experience Rot” – the gradual deterioration of user experience quality from moving too slowly or focusing solely on economic outcomes rather than user needs.

The costs manifest in several ways:

• Opportunity cost: Market opportunities disappear while waiting for test results that may not even be necessary
• Resource waste: Development time gets tied up in prolonged testing initiatives for low-risk changes
• Analysis paralysis: Teams debate endlessly about what to test next instead of making decisions
• Competitive disadvantage: Competitors gain ground while you’re stuck in lengthy validation cycles

The key is matching your experimentation method to the decision you’re making, rather than forcing every design change through the same validation process.

A framework for design validation decisions

The path to better validation starts with two fundamental questions about any proposed design change:

1. Is this strategically important? Does this change significantly impact key business metrics or user experience?
2. What’s the potential risk? What happens if this change performs worse than expected?

Using these dimensions, you can map any design change into one of four validation approaches:

High Strategic Importance + Low Risk = Just ship it

If you can’t explain meaningful downsides to a design change but know it’s strategically important, you probably don’t need to validate it at all. These are your quick wins.

Examples for SaaS teams:

• Adding customer testimonials to your pricing page
• Improving mobile responsiveness
• Fixing broken links or outdated screenshots
• Adding clearer error messages in your product

Why this works: The upside is clear, the downside is minimal, and the time spent testing could be better invested elsewhere.

Low Strategic Importance = Deprioritize

Not every design change needs validation because not every change is worth making. Some modifications simply aren’t worth the time and resources, regardless of the validation method you might use.

Examples of low-impact changes:

• Minor color adjustments to non-critical elements
• Changing footer layouts
• Tweaking secondary page designs that get little traffic
• Adjusting spacing that doesn’t affect usability

When to reconsider: If data later shows these areas are creating friction, they can move up in priority.

High Strategic Importance + High Risk = Validation territory

This is where both A/B testing and rapid testing methods become valuable. The critical next decision becomes: can you reach statistical significance within an acceptable timeframe, and are you technically capable of running the experiment?

When to use A/B testing vs rapid testing

When to use A/B testing for design changes

A/B testing remains your best option for design changes when:

• You have sufficient traffic on the experience: Generally, you need 1,000+ visitors per week to the page being tested
• The change is reversible: You can easily switch back if the results are negative
• You need statistical confidence: Stakes are high enough to justify the time investment
• Technical capability exists: Your team can implement and track the test properly

Examples of SaaS use cases for A/B testing:

• Complete homepage redesigns
• Pricing page layouts and messaging
• Sign-up flow modifications
• Core product onboarding changes
• High-traffic landing page variations

When to use rapid testing for design changes

When A/B testing isn’t right due to traffic constraints, technical limitations, or time pressures, rapid testing provides a faster path to validation.

Rapid testing methods work particularly well for SaaS design validation because they can:

• Validate concepts before development: Test wireframes and mockups before building
• Narrow down options: Compare multiple design variations quickly
• Identify usability issues: Spot problems before they reach real users
• Provide qualitative insights: Understand the “why” behind user preferences

Examples of SaaS use cases for rapid testing:

• New feature naming and messaging
• Dashboard navigation restructuring
• Enterprise sales page designs (low traffic)
• Value proposition clarity testing
• Multi-option comparisons (6-8 variations)

The natural next question might be “which rapid testing method should I use?” Here is another decision tree framework to help answer that.

Incorporate your experimentation strategy into your design process

With a decision-making strategy for how and what to test, you’ll need to incorporate the strategy into your design process. The most successful SaaS teams don’t treat validation as an afterthought. They build it into their process from the beginning:

• During ideation: Use rapid testing to validate concepts and narrow options before detailed design work
• During design: Test wireframes and mockups to identify issues before development
• Before launch: Use A/B testing for high-stakes changes, rapid testing for others
• After launch: Continue testing iterations based on user feedback and performance data

The compounding benefits of a sound experimentation strategy

The goal isn’t to replace A/B testing with rapid methods or vice versa. Both have their place in a mature experimentation strategy. The key is understanding when each approach provides the most value for your specific situation and constraints.

Teams that master this balanced approach to validation see remarkable improvement, including:

• 50% better A/B test win rates (because rapid testing helps identify winning concepts)
• Faster time-to-market for design improvements
• More confident decision-making across the organization
• Better team morale from seeing results from their work more quickly

Perhaps most importantly, they avoid the extremes of either testing nothing (high risk) or testing everything (slow progress).

For SaaS teams serious about optimization, the question isn’t whether to validate design changes; it’s whether you’re using the right validation method for each decision.

Start by auditing your current design change process. Are you testing changes that should be implemented immediately? Are you implementing changes that should be tested? By aligning your validation approach with the strategic importance and risk level of each change, you can move faster without sacrificing confidence in your decisions.

And if you aren’t sure how to get started, our team can help.

The post How to Validate Website Design Changes: A Decision Framework appeared first on The Good.

While 89% of product teams are conducting user interviews according to recent industry data, there’s a critical gap between gathering user input and uncovering the insights that actually drive business results.

We see this all the time in our client work. Teams building features that competitors have without competitor data, or developing features based on the loudest customers without checking the significance of those friction points.

So what’s the solution?

The companies consistently shipping features that move the needle know the difference between asking users what they want and understanding what they actually need. It starts with discovery research.

What is discovery research in UX?

Discovery research in UX is the foundational phase of user research that focuses on understanding user problems, needs, and contexts before any solutions are designed.

Unlike evaluative research methods that test existing designs or prototypes, discovery research explores the unknown territory of user behavior to uncover opportunities and define problems worth solving.

Discovery research helps you understand use cases and user needs. It can ground you in what problems to solve and what is going on in the market.

This grounding is essential for product teams who want to build features that users actually need and will drive growth.

Discovery research typically involves methods like user interviews, field studies, diary studies, and market analysis. These approaches help teams understand the broader context of user goals and challenges before jumping into design solutions. The insights gathered during this phase become the strategic foundation for all subsequent product decisions.

Discovery research versus UX discovery

While these terms are often used interchangeably, there’s an important distinction that affects how product teams approach their research strategy.

Discovery research specifically refers to the research methods and activities used to understand user needs and identify problems. It’s the “how” of gathering insights through interviews, observations, and analysis. This includes techniques like ethnographic studies, user interviews, and competitive analysis.

UX discovery, on the other hand, is the broader strategic phase that encompasses discovery research, but also includes other activities such as technical feasibility assessments, business viability analysis, and stakeholder alignment. UX discovery is the “what and why” that frames the entire early-stage product exploration.

Think of discovery research as the tactical execution within the strategic framework of UX discovery. A comprehensive UX discovery process will include multiple types of discovery research methods. It also considers business constraints, technical limitations, and market opportunities.

For SaaS product teams, this distinction matters because it clarifies roles and expectations. UX researchers lead discovery research activities, while product managers typically orchestrate the broader UX discovery process that incorporates research findings into strategic decisions.

Understanding this difference helps teams avoid the common mistake of treating research as a checkbox activity rather than a strategic input that informs product direction.

Benefits of discovery research

Discovery research delivers tangible benefits that extend far beyond the research team, directly impacting product success and business outcomes.

Reduces development risk and waste

The most immediate benefit of discovery research is risk reduction. By understanding user needs and the specific problems before development begins, teams avoid building features that miss the mark. This is particularly critical for SaaS teams where failed features mean ongoing maintenance costs and technical debt that compound over time.

Enables data-driven product decisions

Discovery research transforms product decisions from opinion-based to evidence-based. Instead of stakeholder preferences driving priorities, user insights guide development resources toward the highest-potential impact opportunities.

Uncover hidden opportunities

Discovery research often reveals unmet user needs that aren’t obvious from analytics or existing feedback channels. These insights can become the foundation for innovative features that differentiate your product in the market.

Improves cross-team alignment

When discovery research findings are shared across product, design, and development teams, everyone gains a shared understanding of user priorities. This alignment reduces conflicting opinions and streamlines the development process.

Accelerates time-to-market for successful features

While discovery research requires upfront time investment, it actually accelerates the development of successful features by ensuring teams build the right things from the start.

Enhances user satisfaction and retention

Products built on solid discovery research foundations better meet user expectations, leading to higher satisfaction scores and improved retention rates. Users feel heard and understood when products solve their actual problems rather than perceived problems.

This is essential for SaaS businesses where discovery research can identify the difference between features that drive daily engagement versus one-time usage, directly impacting churn rates.

When to use discovery research

Discovery research is best leveraged as part of a continuous research strategy.

Teresa Torres, expert and author of Continuous Discovery Habits, recommends weekly conversations with customers. “Continuous discovery means weekly touchpoints with customers by the team building the product, where they conduct small research activities in pursuit of a desired outcome.”

The goal is to take research from something you pause to do, into something you always do.

Many leaders will have experimentation rituals that allow quick and consistent feedback on ideas/products, but it’s rarer to see teams prioritize discovery on a frequent cadence.

When you manage discovery in batches or isolated sprints, it can mean you miss out on opportunities or delay solving urgent problems for customers.

Common discovery activities in UX

Effective discovery research employs multiple methods to build an understanding of the problem landscape and market conditions. Not all are required, but a combination will give a better picture to work off.

Diary studies

For understanding user behavior over time, diary studies ask participants to record their experiences, thoughts, and interactions over days or weeks. This method is particularly valuable for SaaS products where user needs evolve or vary based on different use cases and timeframes.

User interviews

One-on-one conversations with users can be a great pillar of discovery research. The key to successful interviews in discovery is asking open-ended questions that help explore user motivations, frustrations, and workflows. A good foundation is to conduct 6-8 interviews per user segment to get a picture of current challenges and behaviors.

Field studies and contextual inquiry

Observing users in their natural environment provides insights that interviews alone can’t capture. Field studies reveal the environmental, social, and technical factors that influence user behavior, uncovering needs that users might not articulate in interviews.

Competitive analysis and market research

Understanding the competitive landscape helps identify opportunities for differentiation. It also uncovers whether user problems are being adequately solved by existing solutions. This desk research complements user-facing research methods.

Jobs-to-be-done (JTBD) research framework

JTBD research helps frame what job users are “hiring” your product to do. It can help you think beyond features to understand the fundamental progress users are trying to make in their lives or work.

Card sorting

This method helps teams understand how users categorize information and conceptualize problem spaces. Card sorting is particularly useful for discovering how users naturally group features or content areas.

Survey research

While qualitative methods provide depth, surveys can help uncover findings across larger user populations. Use surveys to quantify the prevalence of problems discovered through qualitative research.

Leveraging discovery research for better outcomes

In an era where 83% of designers, product managers, and researchers agree that research should be conducted at every stage of product development, it’s critical to understand discovery research in UX.

Discovery research is a tool that helps you dig into current user needs and prioritize the problems worth solving. It provides the user insights needed to build theme-based roadmaps, prioritize high-impact features, and avoid costly development mistakes. Most importantly, it ensures that every dollar spent on product development addresses real user needs rather than perceived problems.

Ready to make discovery research work for your product team? The Good specializes in helping SaaS companies uncover the user insights that drive product success. Our team combines deep research expertise with practical product strategy to ensure your research translates into features that drive growth.

Get in touch with The Good to discuss how discovery research can accelerate your product development and improve user satisfaction. Let’s turn your user insights into your competitive advantage.

The post What Is Discovery Research in UX? appeared first on The Good.

“The fundamental shift that people have to make is that you’re no longer a data collector. You’re a data connector,” says Ari Zelmanov, former police detective and current research leader. In Ari’s view, as teams get leaner and tools get better at executing research tasks, the job of the researcher becomes standing up repositories, socializing learning mechanisms, and creating the systems that empower organizations to act on good information.

We spoke with research leaders who've successfully made this transition, transforming their teams from siloed specialists into customer-centric learning cultures. Their approaches varied, but one theme was clear: when you empower others to answer their own questions, you don't diminish your value, you multiply it.

The d word holding us back

Before diving into solutions, there's an elephant we need to address: Democratization. Many researchers worry that democratizing research will lead to poor methodologies, incorrect conclusions, or devalued expertise. But Ari feels the argument is nye.

"The only people arguing about democratization are researchers," says Ari. "Nobody else is arguing about it. We're infighting about something that we have zero control over. It's happening."

I tend to feel like anyone arguing about democratization is missing one critical point: customer centricity isn't just one person's job.

Anton Krotov, Researcher in an organization of over 10,000 people, was in the fortunate position of being very trusted by his colleagues. So much so that they believed research could answer all of their questions.

“I had already established a reputation. I was fortunate that I didn't need to sell the value of research. Quite the opposite. People came to me with too many requests. They believed research could do everything for them. I needed to set up boundaries.”

Overwhelmed with requests from colleagues, Anton realized that the solution wasn't saying no—it was saying yes in a different way. Rather than becoming a bottleneck, Anton chose to become a bridge.

Connect teams through shared intelligence

Good intelligence is the responsibility of many disciplines, not just research. To get answers quickly, Ari's teams use what he calls the "Moneyball" approach to research, a framework that prioritizes speed and accessibility over methodological purity:

"Product teams are incentivized to move fast. So, how do you make research fit into that in a way that makes sense? We built something called Moneyball Research. It's super simple: start with what you know. It could be in your repository, it could be what you know. Then you go to what data is accessible within 24 to 48 hours. That's usually internal analytics, CSAT tickets, NPS, sales conversations, and tribal knowledge. Then—and only then—do you go to primary research."

This approach shifts conversations away from methods and focuses instead on what teams need to know and how confident they need to be. "Then it's up to the researcher to be the doctor. Diagnose that, determine how they're going to collect that evidence given the time, money, and level of rigor."

René Bastijans, lead researcher at a growth-stage startup, has found creative ways to loop colleagues into data collection. His sales team is trained to lightly survey prospects during sales calls and report back to the wider team.

"We've trained our sales team to ask for specific data and enter it into Salesforce. Researchers and the product team have access to these data, and therefore, sales has allowed us to keep a pretty good pulse on the market."

This creates a healthy feedback loop that keeps everyone abreast of evolving user needs while extending the research team's reach without expanding headcount.

Invite colleagues into the research process

While it might seem counterintuitive to share methodologies and research responsibilities, successful research leaders see democratization as an opportunity rather than a threat.

To remove research bottlenecks, Anton ran internal workshops to upskill his colleagues on doing their own research. This proactive approach to education focused on tailoring training to his colleagues' specific needs: "I try to cover the cases that will be really applicable, so I don't offer any cookie-cutter material and don't go much into theory. It's really tailored to their day-to-day work."

The key is meeting people where they are and giving them tools that fit their contexts. Not everyone needs to become a master researcher, but many can learn to conduct basic customer interviews or query data effectively.

Brittany Lang, UX Research Manager and M.S. in Information Science, uses project reviews as a time to cultivate a shared point of view and continually refine her thinking.

“Before we socialize research plans, I usually take a look at it, or I have someone else on my team take a look at it. It doesn't have to be your manager that's reviewing something, but can someone give you feedback?

It's nice when coworkers leave comments and I can see what other people on the team have said and we can agree or challenge, and then have a discussion about it. I also learn in those moments too. When I'm looking at how members of my team have reviewed other work, where they're coming from and their perspective, I learn a lot from them in those moments.”

Facilitate low-risk learning

It takes more than a few ambitious researchers to imbue a company’s culture with a learning mindset, which is why rituals and learning programs are so important.

Anton’s employer formalized this approach to building safe learning environments through a program called "Gigs for Growth," a repository of side projects from different departments where employees can apply to work on learning opportunities outside their typical scope.

"It's like a company green light that you can work on learning during your full-time gig and outside of your typical work scope. Something that you would never otherwise be able to touch in the company."

Under this program, researchers can support QA engineers, sales can support marketing, and everyone gets exposure to new perspectives that inform their primary roles. "You get some really new experiences that otherwise you wouldn't be able to."

At The Good, we like to build regular, low-stakes opportunities for knowledge sharing and skill development. One of our approaches at The Good is a ritual called "Random Question of the Week." During another bi-weekly meeting, team members share client questions that stumped them or that they felt they could have answered better.

These conversations help build shared perspectives that then get turned into artifacts:

• FAQ entries for brief, punchy answers
• Articles for long-form perspectives
• Policies or SOPs that outline ways of working

The result is that teams become more aligned, can answer tough questions on the spot, and save time by referring to their collective knowledge instead of rehashing the same discussions.

Another effective ritual is "Critique & Share" sessions, where team members bring questions, websites they admire, or work they're developing to get fresh perspectives from colleagues who haven't been deep in the weeds of a particular project.

Maggie Paveza, Senior Strategist at The Good, shares that it has helped her break the ice when building a shared P.O.V.

"It's pretty informal and often we're not showing our own work, so it feels less intimidating to ask your team members, 'why do you think this competitor is using this strategy,' than if it were your own work," explains Maggie.

The power of being a data connector

"The fundamental problem that research as an industry has is we've been myopically focused on the front end of the equation," says Ari. "Data collection, statistical significance, theoretical saturation—insert whatever fancy academic word you want in here. But the real power comes on the back end of the equation."

That back end is about connection, synthesis, and empowerment. When researchers shift from being data collectors to data connectors, they don't lose their expertise; they amplify it.

As Anton puts it, "Where soil is right, then you can do things. Praise people for when they do things great. You can learn from mistakes, you can learn from success."

The goal isn't to turn everyone into a researcher. It's to create an environment where customer insights flow freely, where good questions get asked by many disciplines, and where learning happens continuously rather than in bursts.

Making the shift

Building a customer-centric learning culture doesn't happen overnight, but it starts with understanding where your organization is open to change and being constructive about how you facilitate it.

Look for teams and individuals who are already curious about customers. Find the places where people are asking good questions but lack the tools or confidence to find answers. Then meet them there with the right combination of education, tools, and support.

"At the end of the day, it's about empowering decision-making," says Ari. And in a world where customer expectations evolve quickly and research teams are lean, that empowerment might be the most valuable thing researchers can provide.

The post From Data Collector to Data Connector: Embracing Research Democratization appeared first on The Good.

Don’t get me wrong, A/B testing is incredibly powerful. It’s the gold standard for high-stakes, high-traffic decisions where statistical significance matters most. But when it becomes your only tool, you create unnecessary constraints that can paralyze decision-making and slow innovation.

The reality is that different decisions require different levels of rigor, confidence, and investment. Luckily, there is a complementary approach that fills critical gaps in your experimentation toolkit. By understanding when each method is most appropriate, teams can make faster, more informed optimizations while maintaining the rigor needed for their most high-stakes decisions.

Creating “experience rot”

A/B testing borrowed its methodology from medical intervention studies, where 95% confidence intervals and statistical significance aren’t just nice-to-haves; they’re life-or-death requirements.

But we’re not rocket scientists, and we don’t always need the same level of assurance in product decisions to move towards the right outcome.

A/B testing can be overkill for the decisions product teams need to make daily. Yet teams have become so committed to this single methodology that they’ve created what researcher Jared Spool calls “Experience Rot,” the gradual deterioration of user experience quality from teams moving too slowly or focusing solely on economic outcomes.

The costs of slow testing cycles are tangible and measurable:

• Market opportunities disappear while waiting for test results
• Competitors gain ground during lengthy testing phases
• Development resources get tied up in prolonged testing initiatives
• Customer frustration builds as issues remain unfixed
• Decision fatigue sets in as teams debate what to test next

But the problem runs deeper than just speed. Many teams face contexts where A/B testing simply isn’t feasible. Regulatory challenges in healthcare and finance, low-traffic scenarios for B2B products, technical constraints, and organizational politics all create barriers to traditional experimentation.

By the time a test idea passes through all the bureaucratic loopholes and oversight at an organization, it’s often no longer lean enough to justify testing. Without an alternative testing method, teams are left without any data at all.

So, how do we:

1. Circumvent the challenges of A/B testing, and
2. Prevent experience rot?

Enter rapid testing

Rapid testing isn’t about cutting corners or accepting lower-quality insights. It’s about matching your research method to the decision you’re trying to make, rather than forcing every question through the same rigorous, but often slow, process.

Like A/B testing, rapid testing helps you understand if your solutions are working. Unlike A/B testing, rapid tests are conducted with smaller sample sizes, completed in days rather than weeks or months, and often provide qualitative insights that A/B tests can miss.

“The speed at which we obtain actionable findings has been impressive,” says Gabrielle Nouhra, Software Director of Product Marketing, who leverages rapid testing with The Good for research and experimentation. “We are receiving rapid results within weeks and taking immediate action based on the findings.”

The key is understanding when each approach makes sense. Not every decision requires the same level of rigor, and smart product teams create systems that allow critical insights to move faster.

A framework for decision making

So, how do you decide when to use rapid testing versus A/B testing? The decision starts with two critical questions: Is this strategically important? And what’s the potential risk? With those two questions in mind, you can map your ideas on a simple 2×2.

High Strategic Importance + Low Risk = Just Ship It. If you can’t explain meaningful downsides to a change but know it’s strategically important, you probably don’t need to test it at all. These are your quick wins.

Low Strategic Importance = Deprioritize. Not everything needs to be tested. Some changes simply aren’t worth the time and resources, regardless of the method you use.

High Strategic Importance + High Risk = Test Territory. This is where both A/B testing and rapid testing live. The next decision point becomes: Can you reach statistical significance within an acceptable timeframe? Are you technically capable of running the experiment?

If the test isn’t technically feasible or traffic constraints make the time-to-significance longer than is acceptable, rapid testing becomes your best option for de-risking the decision.

Rapid testing in practice

Rapid testing encompasses various methodologies, each suited to different types of questions. Here are just a few examples:

First-Click Testing helps confirm where users would naturally click to complete a task. Perfect for interface design decisions and navigation optimization.

Preference Testing goes beyond simple A/B comparisons to evaluate multiple options, often six to eight variations, helping teams understand which labels, designs, or approaches resonate most with their target audience.

Tree Testing reveals where users might stray from their intended path, using nested structures to understand navigation behavior without the distraction of full visual design.

The beauty of these methods lies in their speed and specificity. Rather than testing entire page redesigns, rapid testing allows you to validate specific hypotheses quickly. Which onboarding segments will users self-identify with? Where should we place a new feature to maximize engagement? Which design elements increase trust among new visitors?

Rapid tests can also guide our A/B testing strategy. If we’re entertaining multiple options for new nomenclature within an app experience and we’re just trying to understand which label users think would be most accurate or most likely to represent those outcomes, running a rapid test can narrow down those options and help us decide what to A/B test.

Building a rapid testing practice

Implementing rapid testing effectively requires more than just choosing the right method. Teams that see the best results follow several key principles:

1. Impact pre-mortems: Before testing, clearly define what success looks like and what impact you expect if implemented. This helps connect testing activities to business outcomes and prevents post-hoc justification of results.
2. Acuity of purpose: Keep tests focused on specific questions rather than trying to evaluate everything at once. A/B testing often encourages comprehensive evaluations, but rapid testing works best with precise hypotheses.
3. Pre-defined success criteria: Establish clear benchmarks before you start testing. If 80% of users can complete a task, is that a win? What about 60%? Define these thresholds upfront to avoid moving goalposts when results come in.
4. Mute context: When testing specific elements, remove unnecessary context that might distract from the core question. Full-page designs can overwhelm participants and dilute feedback on the element being tested.
5. Sunlight: Even experienced researchers benefit from collaborative review of test plans. Transparency builds confidence in the process, and a peer review of test designs helps identify potential issues before execution.
6. Share: Circulate your impact, what you’ve learned about your audience, and get people excited about the work. The goal is to build visibility, create a case for why this work is valuable, and encourage people to make decisions with data.

The compound effects of speed

Teams that successfully implement rapid testing alongside their existing A/B testing programs see remarkable results. Our clients report 50% improved A/B test win rates, better customer satisfaction scores, and significantly faster time-to-insights.

But perhaps most importantly, they report better team morale. There’s something energizing about seeing results from your work quickly, about being able to iterate and improve based on real user feedback rather than lengthy committee discussions.

It’s never too late to pivot. The idea is to move from long-term decision making, where we send something through the whole development and design cycle only to come up with a lackluster outcome, to form a process that helps us get quick, early signals.

Making the transition

The goal isn’t to replace A/B testing. It remains the gold standard for high-stakes, high-traffic decisions. But by adding rapid testing to your toolkit, you can accelerate the decisions that don’t require months of statistical validation while still maintaining confidence in your choices.

As decision scientist Annie Duke writes in Thinking in Bets, “What makes a great decision is not that it has a great outcome. It’s the result of a good process.” Rapid testing gives teams a process for rational de-risking that emphasizes both speed and quality.

The question isn’t whether you should test your ideas; it’s whether you’re using the right testing method for each decision. In a world where speed increasingly determines competitive advantage, teams that master this balance will consistently outpace those stuck with only one tool in their kit.

Ready to accelerate your decision-making process? Our team specializes in helping product teams implement rapid testing alongside existing experimentation programs. Get in touch to learn how we can help you cut testing time without sacrificing insight quality.

The post How To Make User-Centered Decisions When A/B Testing Doesn’t Make Sense appeared first on The Good.

Despite this being a proven economic model, companies are rarely focusing their effort on that 20%.

It’s not because they don’t want to; it’s because it is easy to get wrapped up in not losing a single sale, to the point that you are spreading yourself too thin.

If you focus your energy and product improvements on the highest-value user segment, you will see greater returns for less work.

In this article, we’re sharing the study we recently ran for a client that helped us identify their most valuable user segments and prioritize improvements to meet their needs.

What are user segments?

User segments are groups within a customer base who share similar characteristics, behaviors, or values.

They are created with user segmentation, which researches those commonalities and divides your audience into distinct groups. You can then tailor experiences, personalize messaging, and focus optimization efforts on their specific needs.

Common types of user segments

User segments can be divided based on different traits. The type of segmentation you use will vary based on your use case and goals. Here is a quick overview of common user segments.

Why companies optimize for the wrong segments

When we run prioritization exercises, one of the most common mistakes we see is companies focused on segments of users based on volume. If the segment has more users, they automatically believe it deserves more attention.

This reflects one of the three common prioritization mistakes:

1. Volume bias: Prioritizing segments with the most users rather than the most value
2. Squeaky wheel focus: Optimizing for the users who complain the loudest
3. Recency fallacy: Focusing on the latest acquisition channel or user cohort without evaluating their actual value

The uncomfortable truth is that your most valuable segments may not be your largest, your loudest, or your newest.

Conducting a segmentation study step by step

At The Good, we’ve developed a systematic approach to identify and prioritize your most valuable user segments. Here’s how it works.

Step 1: Set your goals

Before you start analyzing data, segmenting users, and prioritizing, you need a clear understanding of your project goals. In most cases, they will look something like this:

• Identify and quantify subsets of user segments based on use cases
• Understand the potential value of known segments
• Identify features and benefits that are most important on a per-segment basis
• Find opportunities to improve the engagement of high-value users

These goals can be turned into the key research questions of your study.

Step 2: Identify valuable behaviors beyond revenue

Your most valuable user segments, of course, need to drive revenue, but there are other indicators to consider when prioritizing who you are building/optimizing for.

Current value metrics, future value indicators, influence value, and cost-to-serve factors will all influence the overall value of a user segment.

• Current value metrics: Revenue generated, subscription tier, feature usage, team size
• Future value indicators: Growth trajectory, expansion potential
• Influence value: Referral behavior, advocacy impact
• Cost-to-serve factors: Support requirements, implementation complexity, churn risk, acquisition cost

Identifying and tracking these metrics and scoring segments based on this information will help paint a more holistic picture of value. Some segments might not be your biggest revenue drivers today, but they represent significant future opportunities, so you may choose to optimize for them instead of your current biggest spenders.

Step 3: Collect qualitative and quantitative data

Once you’re clear on goals and value metrics, you’re ready to start collecting data for your segmentation analysis. Gathering a multidimensional data set will help you better understand users as the complex humans they are. Types of data that will help your analysis will include:

• Usage patterns: Frequency, features used, time spent in the product
• Transactional data: Revenue contribution, plan type, upgrade/downgrade history
• Behavioral signals: Engagement with key activation points, referral behavior
• Acquisition source: Channel origin, customer acquisition cost, time to convert
• Demographic/firmographic data: Company size, industry, role

Most of this data will be sourced from your main quantitative collection tool, such as Google Analytics or your product analytics. But for a truly effective study, you need to supplement all this information with qualitative context. Surveys, session recordings, or user tests can help you better understand why your users are doing what they do.

Step 4: Conduct factor analysis to identify value drivers

Group your data together into a reduced number of independent factors that represent the underlying themes within the dataset. This will help identify value drivers that differentiate your user segments.

For example, in a recent segmentation project, we discovered distinct value factors that formed natural segment groupings:

• Efficiency seekers: Primarily valued time savings and streamlined workflows
• Integration power users: Heavily utilized connections to other tools in their stack
• Data-driven optimizers: Focused on analytics and performance insights
• Scale-focused operators: Needed enterprise features and team collaboration

Understanding these value drivers helps you move beyond simple demographic segmentation to truly understand what motivates different user groups.

Step 5: Apply cluster analysis to form actionable segments

Once you’ve identified the key value drivers, use cluster analysis to group users with similar characteristics. Usually, 3-7 distinct segments emerge from the exercise.

These segments often cross traditional demographic lines, revealing unexpected patterns. For example, power users might not be enterprise customers as you assumed, but mid-market companies with specific workflow needs.

This is also the time to start looking for natural clusters of behavior that indicate high-value segments. Considering this, when you’re analyzing user clusters, look for key differentiators like:

1. Usage frequency: Daily users vs. weekly vs. monthly
2. Feature utilization: Which user flows are most common for each segment
3. Value perception: What features each segment values most highly
4. Growth potential: Which segments show increasing usage over time

Step 6: Quantify segment value and opportunity size

The inputs from your data, factor, and cluster analyses will produce outputs of your high-value segments.

Here’s an example of that workflow so far. The data (survey themes collected) on habits, values, and use cases were the inputs for the factor and cluster analyses. That resulted in segments around the frequency of product use, customer values, and reason for use.

For each potential high-value segment, revisit the value metrics you established in step 2 of the process. Calculate the relevant metrics to ensure you’re not just following hunches but making data-backed decisions about where to focus.

The most valuable segments often show strength across multiple metrics, not just in current revenue. For example, a segment with moderate current revenue but excellent retention and high referral rates may be more valuable than a high-revenue segment with poor retention.

You’ll also start to see how your most valuable segments differ from your hypotheses. Maybe it’s not defined by company size but by a specific usage pattern. As a specific example, imagine users who perform at least 3 exports per week AND invite 2+ team members within the first 30 days are 4.5x more likely to upgrade to the enterprise tier within 6 months.

This kind of insight could transform your priorities, focusing on making these specific actions easier and more intuitive, rather than spending time/money on creating new features for other segments.

Step 7: Map segments to specific opportunities

The final step is to leverage your knowledge about high-value users to focus optimization efforts. Now, you can connect your segment analysis to concrete optimization opportunities. A few thought starters for this process:

1. What actions correlate with long-term success for this segment?
2. Where do users in this segment typically struggle?
3. What capabilities does this segment need but doesn’t have?
4. What value propositions connect most strongly to this segment?

You’ll end up with a list of optimization opportunities. To prioritize those efforts and start building a roadmap, we recommend scoring them across these dimensions on a 1-10 scale, then calculating a weighted score that reflects your company’s specific situation and constraints.

1. Potential revenue impact: How much additional revenue could optimizing for this segment generate?
2. Implementation effort: How difficult would it be to implement changes for this segment?
3. Time to results: How quickly can you expect to see meaningful outcomes?
4. Strategic alignment: How well does focusing on this segment align with your long-term business strategy?

For example, if you’re under pressure to show quick wins, you might weigh “time to results” more heavily. If you’re planning for long-term growth, strategic alignment might carry more weight.

This will be the start of your roadmap for optimization efforts, ensuring that you focus resources on the right opportunities for your most valuable segments.

Focus on your highest-value segments first, then gradually expand your optimization efforts to secondary segments once you’ve captured the initial value. Always consider potential cross-segment impacts when making changes.

Drive growth with user segmentation and prioritization

As your product and market evolve, so will your user segments. What constitutes a high-value segment today may shift as you introduce new features or enter new markets.

We recommend evaluating your user segments quarterly, with a more comprehensive review annually or whenever you experience significant business changes.

Remember, the path to scaling your SaaS business isn’t through trying to please everyone with generic optimizations. It’s through deeply understanding which user segments create the most value and deliberately focusing your limited resources on enhancing their experience.

Ready to identify and prioritize your most valuable user segments? The Good’s Digital Experience Optimization Program can help you discover untapped growth opportunities through expert research, strategic insight, and data-driven experimentation. Contact us to learn more about how our team can help your SaaS business scale faster.

The post How to Identify Your Most Valuable User Segments and Prioritize Accordingly appeared first on The Good.

But this episodic style is falling out of favor with forward-thinking teams. The best product organizations are embracing continuous research, an always-on approach to gathering insights that allows for more user-centered, effective products.

In one study, 83% of designers, product managers, and researchers agreed that research should be conducted at every stage of the product development life cycle. But, only 36% of them are conducting research studies after launch.

How can they bridge the gap? With continuous research.

What is continuous research?

Continuous research is an “always-on” style of research, where product teams put practices and systems in place to habitually learn from users. Rather than conducting isolated research studies or sprints, it focuses on integrating regular research activities into the product development cycle.

Why continuous research?

The benefits of continuous research are plentiful. Gathering insights regularly means quickly responding to user needs/wants, making more data-driven decisions, and reducing spend on changes that don’t work.

According to research by McKinsey, there is a direct correlation between financial success and de-risking development by continually listening, testing, and iterating with end-users. Continuous research methods are proven to positively impact the bottom line, and you can feel good knowing that they also make your customers’ lives better.

However, the under-touted benefit of continuous research is that it makes everyone’s job at your company easier. Product teams get their questions answered faster. Developers don’t have to waste time on unfriendly user features. Sales can more easily connect with customers.

No one has to wait to get on a roadmap, because there is a constant cycle of feedback and user connection that is otherwise unattainable.

Continuous research methods

So, what specifically counts as continuous research? Plenty of methods would fall under this umbrella. Here are a few of our favorites to paint a picture of what continuous research looks like in action.

Regular user interviews

Open a time on your calendar to consistently fill with customer interviews. This consistent, lightweight user research can gather immediate feedback on new features or designs.

Regular usability testing

Find time to observe users interacting with your product. Do this often, and you will uncover patterns to improve your UX.

Ongoing collection of CSAT or NPS scores

Keep a record of customer satisfaction scores (CSAT) or Net Promoter Scores (NPS) to understand over time whether users are happy with your product. This consistent record will help you determine if product optimizations have helped or hurt your experience.

Cohort comparison through onboarding surveys

Conduct onboarding surveys and then compare cohorts over time to identify trends that may not be apparent in individual feedback sessions.

Lightweight prototype testing

Get feedback on designs from initial prototype to mid-fidelity to fully mocked up. Use the consistent feedback to iterate quickly and make immediate changes as you go.

Continuous research implementation strategies

With the benefits and methods clear, you might be ready to shift your culture towards continuous research. If so, here are a few implementation strategies to set you off on the right foot.

Start small and build up consistency

Begin with a single recurring research touchpoint, such as weekly user interviews or bi-weekly prototype testing sessions. You don’t need a comprehensive, always-on strategy when you’re just starting out. Starting small will get you into the habit, and then you can find ways to expand your efforts.

Put research on the calendar

Blocking time on the calendar for research will get you into the continuous research habit. Consider something like a Friday afternoon standing meeting that you can fill with customer interviews.

Teresa Torres, a well-respected expert and author of Continuous Discovery Habits, suggests you talk to customers every week. “Continuous discovery means weekly touchpoints with customers by the team building the product, where they conduct small research activities in pursuit of a desired outcome.”

The emphasis is on taking research from something you pause to do, into something you always do. Putting it on the calendar in a consistent cadence will help you stick to it.

Get the whole team involved

One of the best parts of continuous research is that it benefits the whole team… and the whole team can be involved! While continuous research may be led by a researcher, it can also be effectively led by product managers who incorporate it into their regular schedule.

Even if other teams don’t lead the process, get them involved by:

• Asking salespeople to ask one specific research question in each call
• Having designers build prototype testing into their workflows
• Sharing research findings across the organization

We have lots of expert insights on how to make B2B research work harder and get your team involved in the process, here.

Complement ongoing feedback with strategic research

Another great recommendation from Teresa Torres is to complement ongoing feedback with occasional deeper discovery work. When you have a higher-risk change or question, take the necessary time to do a deep dive into the data, testing, and analysis.

An always-on research strategy should ensure you’re solving the right problems and that you’re doing it effectively. A combination of lighter, continuous, and deep-dive research will make sure that happens.

Build your toolkit

Tools and technologies that enable continuous feedback will be a lifesaver during busy weeks when it would be easy to let research fall to the wayside. Set up automations and find the tools that make it easier for you to keep users scheduling, data collecting, and insights surfacing.

In the end, the value of continuous research comes from rapidly applying insights. These implementation strategies will create explicit pathways for research findings to influence product decisions within days, not months.

Who should leverage continuous research?

A shift to continuous research represents a cultural shift in product development. It’s not just a changing methodology; it’s a truly user-centered approach where customer feedback continuously shapes product direction. Most product teams would benefit from implementing an always-on research strategy, but it’s particularly valuable in a few circumstances.

Teams with limited resources

It might seem counterintuitive, but it is particularly valuable for teams that don’t have a big research budget. Even without the dollars to fund big studies, teams can leverage continuous research to uncover customer insights that guide development.

Growth-stage startups

It’s ideal for startups that are moving quickly to build and make decisions. They’re mostly throwing things at the wall to see what sticks, but continuous research can act as the safety net or gut check for those ideas. Run it by a customer and get some quick feedback instead of waiting to make mistakes in-market.

Products in rapidly evolving markets

If you’re in a market that is changing quickly, like AI, it’s a good idea to implement continuous research. It helps you adapt to evolving consumer needs more efficiently and to keep up with rapidly developing technological advancements. When you have an always-on research schedule, you can get your questions answered more quickly and implement changes shortly after.

Why “always-on” should be your new normal

Studies show that the compartmentalization of design, development, and research stages of product development “increases the risk of losing the voice of the consumer or of relying too heavily on one iteration of that voice.” Don’t let your organization fall into this trap.

User insights help teams innovate faster and build better products. The best teams today are those that learn from their customers as they build, putting the user experience at the center of product development and optimization. Consistent feedback loops allow them to deliver constant value and effectively respond to market changes.

As competition intensifies in SaaS, continuous research could be the difference between products that thrive and those that die.

If your team sees the value of continuous research but doesn’t have the resources to manage it in-house, The Good can help.

Our team of experts will be an on-demand research (and design and strategy) team that helps you get things done faster. No more waiting months to get your ideas on the roadmap.

The post Continuous Research: The Secret Weapon For Effective Product Teams appeared first on The Good.