Understanding User Behavior Through Testing

A/B testing is not a new method at all, in behavioral science, this logic has been around for a very long time, even if we do not usually call it A/B testing. We call it experiments, condition comparisons, randomized studies, or controlled designs. The basic idea is the same: compare versions, observe responses, and learn from the difference. But in behavioral science, that comparison was never restricted to metrics alone. Depending on the question, researchers have long used quantitative data, qualitative data, or a combination of both to understand how people respond to different conditions. That broader view makes a lot of sense to me, especially because human behavior is rarely just a number. In UX, though, A/B testing is still usually framed in a much narrower way. Most of the time, it is treated as a purely quantitative exercise: which version got more clicks, which flow converted better, which option increased engagement. That is valuable, and I am not arguing against it. I use behavioral metrics too, and they matter. But across projects, one thing has become very clear to me: a metric can tell you that something changed without telling you what that change actually meant to users. I have seen cases where a version performed better, but the real insight was not the lift itself. The real insight was that users felt less hesitation, understood the next step faster, or trusted the experience more. Without talking to them, observing them, or giving them space to explain their thinking, that deeper layer would have remained invisible. That is why I think qualitative methods can make A/B testing far more insightful. Interviews, think aloud sessions, usability observation, and even short open ended follow ups can reveal the mechanism behind the outcome. Instead of stopping at Version B won, we can start understanding whether it reduced confusion, lowered cognitive effort, aligned better with expectations, or made the interface feel more credible. To me, that is where the real value is. A/B testing should not only help us choose between options. It should help us learn something meaningful about perception, attention, trust, decision making, and friction. Otherwise, we risk becoming very good at measuring outcomes while staying relatively shallow in how we interpret them. I also think qualitative methods are useful at more than one stage of the process. Before a test, they help generate stronger variants because the changes are grounded in actual user problems rather than assumptions. During a test, they can capture reactions that behavioral logs cannot fully explain. After a test, they help interpret both positive and null results. Sometimes a version does not win because the change was weak, because users did not notice it, or because the thing the team cared about was not what users cared about. Those are important lessons, and they rarely come from the dashboard alone. Perceptual User Experience Lab

Most teams are just wasting their time watching session replays. Why? Because not all session replays are equally valuable, and many don’t uncover the real insights you need. After 15 years of experience, here’s how to find insights that can transform your product: — 𝗛𝗼𝘄 𝘁𝗼 𝗘𝘅𝘁𝗿𝗮𝗰𝘁 𝗥𝗲𝗮𝗹 𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀 𝗳𝗿𝗼𝗺 𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗥𝗲𝗽𝗹𝗮𝘆𝘀 𝗧𝗵𝗲 𝗗𝗶𝗹𝗲𝗺𝗺𝗮: Too many teams pick random sessions, watch them from start to finish, and hope for meaningful insights. It’s like searching for a needle in a haystack. The fix? Start with trigger moments — specific user behaviors that reveal critical insights. ➔ The last session before a user churns. ➔ The journey that ended in a support ticket. ➔ The user who refreshed the page multiple times in frustration. Select five sessions with these triggers using powerful tools like @LogRocket. Focusing on a few key sessions will reveal patterns without overwhelming you with data. — 𝗧𝗵𝗲 𝗧𝗵𝗿𝗲𝗲-𝗣𝗮𝘀𝘀 𝗧𝗲𝗰𝗵𝗻𝗶𝗾𝘂𝗲 Think of it like peeling back layers: each pass reveals more details. 𝗣𝗮𝘀𝘀 𝟭: Watch at double speed to capture the overall flow of the session. ➔ Identify key moments based on time spent and notable actions. ➔ Bookmark moments to explore in the next passes. 𝗣𝗮𝘀𝘀 𝟮: Slow down to normal speed, focusing on cursor movement and pauses. ➔ Observe cursor behavior for signs of hesitation or confusion. ➔ Watch for pauses or retracing steps as indicators of friction. 𝗣𝗮𝘀𝘀 𝟯: Zoom in on the bookmarked moments at half speed. ➔ Catch subtle signals of frustration, like extended hovering or near-miss clicks. ➔ These small moments often hold the key to understanding user pain points. — 𝗧𝗵𝗲 𝗤𝘂𝗮𝗻𝘁𝗶𝘁𝗮𝘁𝗶𝘃𝗲 + 𝗤𝘂𝗮𝗹𝗶𝘁𝗮𝘁𝗶𝘃𝗲 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 Metrics show the “what,” session replays help explain the “why.” 𝗦𝘁𝗲𝗽 𝟭: 𝗦𝘁𝗮𝗿𝘁 𝘄𝗶𝘁𝗵 𝗗𝗮𝘁𝗮 Gather essential metrics before diving into sessions. ➔ Focus on conversion rates, time on page, bounce rates, and support ticket volume. ➔ Look for spikes, unusual trends, or issues tied to specific devices. 𝗦𝘁𝗲𝗽 𝟮: 𝗖𝗿𝗲𝗮𝘁𝗲 𝗪𝗮𝘁𝗰𝗵 𝗟𝗶𝘀𝘁𝘀 𝗳𝗿𝗼𝗺 𝗗𝗮𝘁𝗮 Organize sessions based on success and failure metrics: ➔ 𝗦𝘂𝗰𝗰𝗲𝘀𝘀 𝗖𝗮𝘀𝗲𝘀: Top 10% of conversions, fastest completions, smoothest navigation. ➔ 𝗙𝗮𝗶𝗹𝘂𝗿𝗲 𝗖𝗮𝘀𝗲𝘀: Bottom 10% of conversions, abandonment points, error encounters. — 𝗕𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗮 𝗖𝗼𝗻𝘀𝗶𝘀𝘁𝗲𝗻𝘁 𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗥𝗲𝗽𝗹𝗮𝘆 𝗣𝗿𝗮𝗰𝘁𝗶𝗰𝗲 Make session replays a regular part of your team’s workflow and follow these principles: ➔ Focus on one critical flow at first, then expand. ➔ Keep it routine. Fifteen minutes of focused sessions beats hours of unfocused watching. ➔ Keep rotating the responsibiliy and document everything. — Want to go deeper and get more out of your session replays without wasting time? Check the link in the comments!

💡 Mapping user research techniques to levels of knowledge about users When doing user research, it's important to choose the right methods and tools to uncover valuable insights about user behavior. It's possible to identify 3 layers of user behavior, feelings, and thoughts: 1️⃣ Surface level - Say & Think This level captures what users say in conversations, interviews, or surveys and what they think about a product, feature, or experience. It reflects their stated opinions, thoughts, and intentions. Example: "I prefer simple products" or "I think this app is easy to use." Methods: Interviews, Questionnaires. These methods capture stated thoughts and opinions. However, insights may be influenced by social norms or biases. 2️⃣ Mid-level - Do & Use This level reflects what users actually do when interacting with a product or service. It emphasizes actions, usage patterns, and observed behaviors, revealing insights that may differ from what users say. Example: Users may claim they enjoy customizing app settings, but data shows they rarely change default options. Methods: Usability Testing, Observation. Observation helps to reveal gaps between what people say and what they actually do. 3️⃣ Deep level - Know, Feel and Dream This level uncovers deep motivations, emotions, desires, and aspirations that users may not be consciously aware of or may struggle to articulate. It also includes tacit knowledge—things people know intuitively but find hard to express. Example: A user might not realize that their preference for a minimalist design comes from the information overload of a current design. Methods: Probes (e.g., participatory design, diary studies). Insights collected using these methods will uncover implicit and emotional drivers influencing behavior. 📕 Practical recommendations for mapping ✅ Triangulate insights by using multiple methods. What people say (interviews/surveys) may differ from what they do (observations) and feel. That's why it's essential to interpret these results in context. For example, start with interviews to learn what users say. Follow up with usability testing to observe real behavior. Use probes for long-term or emotional insights. ✅ Align research with business goals. For product improvements, focus on usability testing to catch interaction issues. For innovation, use probes to generate new ideas from user insights. ✅ Practice iterative learning. Apply surface techniques (like surveys) early to refine assumptions and guide more in-depth research later. Use deep techniques (like probes) for strategic decisions and to foster innovation in long-term projects. 🖼️ UX Research methods by Maze #ux #uxresearch #design #productdesign #uxdesign #ui #uidesign

User research is great, but what if you do not have the time or budget for it........ In an ideal world, you would test and validate every design decision. But, that is not always the reality. Sometimes you do not have the time, access, or budget to run full research studies. So how do you bridge the gap between guessing and making informed decisions? These are some of my favorites: 1️⃣ Analyze drop-off points: Where users abandon a flow tells you a lot. Are they getting stuck on an input field? Hesitating at the payment step? Running into bugs? These patterns reveal key problem areas. 2️⃣ Identify high-friction areas: Where users spend the most time can be good or bad. If a simple action is taking too long, that might signal confusion or inefficiency in the flow. 3️⃣ Watch real user behavior: Tools like Hotjar | by Contentsquare or PostHog let you record user sessions and see how people actually interact with your product. This exposes where users struggle in real time. 4️⃣ Talk to customer support: They hear customer frustrations daily. What are the most common complaints? What issues keep coming up? This feedback is gold for improving UX. 5️⃣ Leverage account managers: They are constantly talking to customers and solving their pain points, often without looping in the product team. Ask them what they are hearing. They will gladly share everything. 6️⃣ Use survey data: A simple Google Forms, Typeform, or Tally survey can collect direct feedback on user experience and pain points. 6️⃣ Reference industry leaders: Look at existing apps or products with similar features to what you are designing. Use them as inspiration to simplify your design decisions. Many foundational patterns have already been solved, there is no need to reinvent the wheel. I have used all of these methods throughout my career, but the trick is knowing when to use each one and when to push for proper user research. This comes with time. That said, not every feature or flow needs research. Some areas of a product are so well understood that testing does not add much value. What unconventional methods have you used to gather user feedback outside of traditional testing? _______ 👋🏻 I’m Wyatt—designer turned founder, building in public & sharing what I learn. Follow for more content like this!