usability-testing

$2a

Small samples reveal big friction

Melanie Perkins: "It's amazing how you can find 10 random people on the internet and they can give such astute feedback that's so representative for such a large number of people." Run tests with as few as 10 random people to identify core product issues.

Watch users, don't just ask them

Uri Levine: "Simply watch users and see what they're doing. If they're not doing what you expect, then ask them why." Direct observation reveals behaviors and needs that surveys miss. Ask 'why' when users deviate from the expected path.

Test multiple options, not one

Kristen Berman: "We never do a UX study where we're just showing people one thing. We always present multiple options and relatively look for which one drives the intended behavior." Single-design testing is ineffective for predicting behavior.

Overcome creator bias

Guillermo Rauch: "You tend to overrate how well your products work. It's very important to give your product to another person and watch them interact with it." Directly observing users helps overcome the tendency to think your product is more intuitive than it is.

Micro-level testing drives millions

Judd Antin: "We changed seven characters and made Airbnb millions of dollars because we found out the button felt scary." Don't dismiss usability testing as junior work; finding scary or confusing CTAs can massively impact conversion.

Progress through testing stages

Itamar Gilad: "Mid-level tests are about building a rough version - early adopter programs, alphas, longitudinal user studies, and fish food (testing on your own team)." Use a progression from fish fooding to dogfooding to alphas to increase confidence iteratively.

Make testing a team sport

Noah Weiss: "We had PMs, engineers, designers, and the user researcher all in one Slack thread live, responding and reacting to the usability session." Increase engagement by having cross-functional teams live-react to sessions in shared chat threads.

Questions to Help Users

• "What specific behavior are you trying to observe or validate?"

• "Do you need to validate the concept (use fake doors) or optimize the execution (use the real product)?"

• "How will you recruit users who have 'zero skin in the game' for honest feedback?"

• "Are you testing one option or multiple options to compare?"

• "What will you do with the findings - how will they flow back into development?"

• "Who else on the team should observe these sessions?"

Common Mistakes to Flag

• Testing only one design - Present multiple options to measure relative performance

• Building before validating - Use Wizard of Oz or fake door tests before writing production code

• Relying on internal intuition - Employees are too familiar with the product to spot real user friction

• Ignoring micro-level issues - Small copy changes and button labels can have massive business impact

• Testing in isolation - Bring engineers and designers into sessions to build shared understanding

Deep Dive

For all 14 insights from 11 guests, see references/guest-insights.md

Related Skills

• Customer Research

• Writing PRDs

• Shipping Products

• Designing Growth Loops