Comparative Usability Studies

NVDA vs VoiceOver For a long time, I thought screen readers were more or less the same. Like many accessibility specialists working on Windows, I mostly used NVDA - it felt efficient, reliable and I thought that it would work everywhere. Digging deeper into real screen reader usage and especially listening to how blind developers actually work changed my perspective. I was surprised how different they feel: 1. NVDA is fast, compact, and very direct. Its default voice (eSpeak NG) is not pretty, but it’s efficient. Many advanced users value speed and clarity over natural sound and NVDA delivers that. 2. VoiceOver feels like a completely different experience. Apple’s voices are more natural, but what really struck me is how much sound design is part of navigation. When moving through code, landmarks, or interface elements, VoiceOver uses audio cues - even subtle musical notes - to signal structure and position: it helps to feel where you are in the content. More of the research: 1. A research comparing NVDA and VoiceOver interactions with interactive controls and structured content (such as tables and form elements) indicate that VoiceOver may read structural cues (like blank table cells and coordinate information) more explicitly than NVDA, which can affect how quickly and confidently a user understands complex layouts. The link to the research: https://lnkd.in/eVbP6PjE 2. Technical evaluations such as the PowerMapper screen reader reliability tests reveal that NVDA and VoiceOver score differently across browser combinations: for example, NVDA paired with Firefox performs very reliably, while VoiceOver on macOS and iOS also shows strong but not identical results in ARIA/HTML tests. The link to the study: https://lnkd.in/eSFACdin 3. A controlled usability research in specific contexts, such as evaluating screen reader performance on job application forms, suggests that NVDA may complete more tasks but require more work-arounds by the user, while VoiceOver users sometimes report higher satisfaction per task even if task outcomes are similar, hinting at differences in effort and perceived efficiency. The link to the research: https://lnkd.in/epjjA3fP So going into this subject I understood that the research and practice don’t support the idea that one screen reader is universally more efficient than the other. NVDA often appears more efficient for speed-focused, keyboard-driven work, especially on Windows. But efficiency is about understanding what each of them reveal. I don’t think research alone gives a full picture of how these screen readers work for real projects. Over the next weeks, I’ll run the same websites through both NVDA and VoiceOver and share short videos showing the practical differences - where issues appear, how navigation feels, and where each tool helps the most. Which screen reader do you use for web accessibility checks? #WebAccessibility #ScreenReaders #NVDA #VoiceOver #AccessibilityTesting #InclusiveDesign

Are you using counterbalance plans in your usability studies? When comparing multiple interfaces, the order in which participants complete tasks can influence results. Counterbalancing helps reduce bias from learning effects or fatigue by varying that order. We recently ran a study on a single feature across four websites. To ensure fair comparison, we used a counterbalance plan to change the order each participant saw the interfaces. Without this, early experiences could skew perceptions of what comes next. Counterbalancing is especially useful when: - Comparing designs or competitors - Measuring preferences across variants - Testing performance differences You can use simple randomization, Latin square design, or a full counterbalance plan. And if you need one quickly, AI can generate it in seconds 💡

💎 Overview of 70+ UX Metrics Struggling to choose the right metric for your UX task at hand? MeasuringU maps out 70+ UX metrics across task and study levels — from time-on-task and SUS to eye tracking and NPS (https://lnkd.in/dhw6Sh8u) 1️⃣ Task-Level Metrics Focus: Directly measure how users perform tasks (actions + perceptions during task execution). Use Case: Usability testing, feature validation, UX benchmarking. 🟢 Objective Task-Based Action Metrics These measure user performance outcomes. Effectiveness: Completion, Findability, Errors Efficiency: Time on Task, Clicks / Interactions 🟢 Behavioral & Physiological Metrics These reflect user attention, emotion, and mental load, often measured via sensors or tracking tools. Visual Attention: Eye Tracking Dwell Time, Fixation Count, Time to First Fixation Emotional Reaction: Facial Coding, HR (heart rate), EEG (brainwave activity) Mental Effort: Tapping (as proxy for cognitive load) 2️⃣ Task-Level Attitudinal Metrics Focus: How users feel during or after a task. Use Case: Post-task questionnaires, usability labs, perception analysis. 🟢 Ease / Perception: Single Ease Question (SEQ), After Scenario Questionnaire (ASQ), Ease scale 🟢 Confidence: Self-reported Confidence score 🟢 Workload / Mental Effort: NASA Task Load Index (TLX), Subjective Mental Effort Questionnaire (SMEQ) 3️⃣ Combined Task-Level Metrics Focus: Composite metrics that combine efficiency, effectiveness, and ease. Use Case: Comparative usability studies, dashboards, standardized testing. Efficiency × Effectiveness → Efficiency Ratio Efficiency × Effectiveness × Ease → Single Usability Metric (SUM) Confidence × Effectiveness → Disaster Metric 4️⃣ Study-Level Attitudinal Metrics Focus: User attitudes about a product after use or across time. Use Case: Surveys, product-market fit tests, satisfaction tracking. 🟢 Satisfaction Metrics: Overall Satisfaction, Customer Experience Index (CXi) 🟢 Loyalty Metrics: Net Promoter Score (NPS), Likelihood to Recommend, Product-Market Fit (PMF) 🟢 Awareness / Brand Perception: Brand Awareness, Favorability, Brand Trust 🟢 Usability / Usefulness: System Usability Scale (SUS) 5️⃣ Delight & Trust Metrics Focus: Measure positive emotions and confidence in the interface. Use Case: Branding, premium experiences, trust validation. Top-Two Box (e.g. “Very Satisfied” or “Very Likely to Recommend”) SUPR-Q Trust Modified System Trust Scale (MST) 6️⃣ Visual Branding Metrics Focus: How users perceive visual design and layout. Use Case: UI testing, branding studies. SUPR-Q Appearance Perceived Website Clutter 7️⃣ Special-Purpose Study-Level Metrics Focus: Custom metrics tailored to specific domains or platforms. Use Case: Gaming, mobile apps, customer support. 🟢 Customer Service: Customer Effort Score (CES), SERVQUAL (Service Quality) 🟢 Gaming: GUESS (Game User Experience Satisfaction Scale) #UX #design #productdesign #measure