For most product teams, the “rigor” of user research is measured by the presence of a visible process. We point to the screener criteria, the structured discussion guide, the target sample size of eight to twelve participants, and the subsequent affinity map as evidence that the resulting insights are valid. There is a profound professional comfort in this procedural compliance; if the steps were followed, the data must be reliable.
However, evidence emerging from the UXinsight 2026 conference suggests that this comfort is a facade. By intersecting UX research methods behavioral science, it becomes clear that we are frequently asking participants to perform cognitive tasks that are biologically impossible. We are then using AI tools to synthesize this meaningless data, structurally amplifying the noise and presenting it as “scalable insight.”
The Myth: Clean Data Equals Valid Insight
The industry has long operated under the assumption that the quality of an insight is proportional to the structure of the method used to extract it. This is the “Research Theater” model. In this framework, a study’s validity is defended through the sheer volume of artifacts: recruitment spreadsheets, meticulously timed interview slots, and Likert scales that transform subjective feelings into quantitative-looking charts.
The danger here is the conflation of procedural rigor with epistemic rigor. Procedural rigor is about following the rules of the craft—making sure the recording is clear and the incentive is paid. Epistemic rigor is about whether the method is actually capable of producing the truth.
When stakeholders demand “statistically significant” qualitative findings, they are pushing researchers toward a version of rigor that prioritizes the appearance of certainty over the accuracy of the observation. This creates a systemic incentive to ask “clean” questions—those that produce easy-to-categorize answers—rather than “true” questions, which are often messy, contradictory, and difficult to map in a slide deck.
Evidence: How UX Research Methods Behavioral Science Exposes the Impossible
The core tension revealed at UXinsight 2026 is that many of our standard research probes violate fundamental principles of cognitive science. Vidhika Bansal’s findings highlight a recurring failure in how we frame questions: we frequently ask participants to simulate future behavior, explain the “why” behind unconscious decisions, or predict their reactions to hypothetical scenarios.
From a behavioral science perspective, these are cognitively impossible tasks. Human beings are notoriously poor at predicting their future selves (the “end-of-history illusion”) and even worse at rationalizing the immediate, intuitive impulses of System 1 thinking.
The Confabulation Problem
When a researcher asks, “Why did you click that button?” they are rarely getting a report of a conscious decision. Instead, they are witnessing confabulation. Because the actual decision happened in milliseconds—driven by pattern recognition and habit—the participant’s brain generates a plausible-sounding rationalization after the fact to satisfy the interviewer.
The participant isn’t lying; they genuinely believe their fabricated reason is the truth. When we treat these rationalizations as “user needs” or “behavioral drivers,” we aren’t designing for the user’s actual mental model—we are designing for the stories they tell themselves about their behavior.
The False Precision of the Scale
This is why the reliance on Likert scales and Net Promoter Scores (NPS) is so problematic. These tools flatten rich, contextual behavioral data into a numerical value. They force a user to quantify a feeling that is likely fluctuating based on their current mood, the lighting in the room, or a recent unrelated frustration.
We treat a “7 out of 10” as a data point, when in reality, it is a low-resolution approximation of a complex emotional state that the user cannot accurately quantify in real-time.
Evidence: AI Tools Are Structurally Incapable of Research Accuracy
As teams feel the pressure to accelerate, the rush to integrate LLMs into the research pipeline has been framed as a way to “scale” insights. However, the technical architecture of these models makes them fundamentally ill-suited for the primary goal of research: identifying the signal within the noise.
Colman Walsh noted that LLMs operate by predicting the most probable next token based on a probability distribution. In essence, AI averages. But research is not about the average; it is about the edge case, the friction point, and the divergent perspective. When an AI summarizes twenty interviews, it structurally tends to smooth over the contradictions and highlight the consensus, erasing the very “outliers” that usually lead to the most significant product breakthroughs.
Confirmation Bias at Scale
Jason Giles pointed out another structural flaw: AI outputs skew toward being agreeable and optimistic. Because they are trained to be helpful assistants, LLMs often reflect the researcher’s own biases back to them. If a prompt is framed as “Find the evidence that users love the new navigation,” the AI will efficiently find the fragments of data that support that narrative, effectively automating confirmation bias.
This is compounded by what Daniel Korczynski identifies as a lack of theoretical grounding. An AI can cluster keywords, but it cannot distinguish between a user’s surface-level complaint and a deep-seated systemic frustration because it does not understand the human experience of frustration. It recognizes the pattern of the word, not the nature of the problem.
The Propagation of Stale Insights
Perhaps the most dangerous trend is the use of RAG (Retrieval-Augmented Generation) systems that allow AI to “learn” from an organization’s internal research repository. Domina Kiunsi warned that this creates a feedback loop of outdated information.
When AI tools draw from a repository containing deprecated personas, superseded findings from three years ago, and defunct product strategies, they propagate these errors at scale. Instead of the repository being a living document, it becomes a source of “automated truth” that anchors the team to obsolete insights. This risk is echoed in recent discussions on the challenges of synthetic users, where training data bias fundamentally skews research outcomes.
The Reframe: ‘Fast and Light’ Can Be More Rigorous Than Traditional Process
Amidst the critique of traditional methods and AI failures, a counter-narrative has emerged. Practitioners like Nidhi Jalwal and Serena Westra demonstrated that “fast and light” research—when grounded in behavioral evidence—can be significantly more rigorous than a massive, procedurally “correct” study.
The discomfort in the room at UXinsight occurred when teams presenting findings from only five interviews, combined with actual behavioral data (logs, session recordings, and observed failure rates), produced more actionable and accurate results than teams who conducted thirty sessions but relied on self-reported “why” questions.
This suggests a necessary redefinition of rigor. Rigor is not a checklist of steps; it is the strength of the evidence chain. An evidence chain looks like this:
- Observation: We saw 40% of users stall at Step 3 (Hard data).
- Inference: We suspect the terminology is ambiguous (Hypothesis).
- Validation: We ran a targeted cloze test to see if users could define the terms (Direct behavioral evidence).
This chain is far more rigorous than a study that says, “We interviewed 20 people, and 12 of them said they found the terminology confusing.” The former relies on what users did; the latter relies on what users think they think. This shift toward evidence-based validity mirrors the broader move toward systems thinking in product design, where the focus shifts from isolated artifacts to the underlying mechanisms of a problem.
Implications: What This Changes for Research Practice
For senior designers and researchers, this evidence requires a shift in how we commission and evaluate research. If we accept that self-reporting is a flawed proxy for behavior and that AI is an “averaging machine,” our tactics must change.
Stop Asking the Impossible
We must stop asking participants to be fortune tellers or psychologists. Instead of asking “Would you use this feature?” or “Why did you do that?”, we should move toward:
- Task-based observation: “Try to complete [X] goal using the current interface.”
- Critical incident technique: “Tell me about the last time you actually struggled with [X].”
- Comparative preference: “Between Option A and Option B, which one lets you finish the task faster?”
Audit AI for “Agreeableness”
Before adopting an AI research tool, we must test it for its tendency to average. Feed it data with a clear, strong outlier and see if the outlier survives the summary. If the AI smooths the contradiction into a general “users had mixed feelings,” the tool is destroying your signal. This is critical when building a trustworthy AI UI, where the system must handle nuance rather than just providing agreeable averages.
Version Control for Insights
To prevent the AI propagation of stale data, organizational repositories must move away from “static PDF reports” toward a version-controlled system. Insights should have explicit expiration dates. When a product pivot occurs, associated research tags should be marked as “superseded” so that RAG systems do not treat 2023 assumptions as 2026 facts.
Defending “Fast and Light”
The most difficult shift is cultural. Defending a small-sample study to stakeholders requires moving the conversation away from sample size and toward evidence strength. Instead of saying “We only talked to five people,” the narrative should be: “We observed the failure in the wild and validated the cause with five targeted tests, which provides a higher confidence level than a large-scale survey of opinions.”
The Uncomfortable Question: What Rigor Actually Looks Like Now
We are facing an inverse relationship between research theater and evidence strength. The teams that look the most “rigorous”—those with the biggest sample sizes, the most polished AI-generated reports, and the most extensive documentation—may actually be producing the least valid insights because they are automating the collection of confabulations.
The discomfort surfacing in the design community signals a necessary paradigm shift. We have to stop valuing the process of searching and start valuing the validity of the find. This is a challenge of cognitive load and theoretical grounding, similar to the “illusion of thinking” observed in large language model reasoning, where a polished output masks a failure in actual logic.
The next time you review a research plan, ask one simple, uncomfortable question: “Is this question asking the participant to do something their brain is actually capable of doing?”
If the answer is no, no amount of AI synthesis or sample size will make the resulting data true.
