You really like this, don’t you?

🌈 Abstract

The article discusses the pitfalls of asking leading questions during qualitative user research, particularly the question "Do you like this?". It highlights how such questions can introduce bias and skew the data, leading to unreliable insights for design decisions. The article emphasizes the importance of using open-ended questions and combining qualitative and quantitative research methods to gather more objective and reliable user feedback.

🙋 Q&A

[01] The Dangers of "Do You Like It?" Questions

1. What are the key issues with researchers asking users "Do you like it?" questions during qualitative research?

• Participants are likely to provide biased, positive responses due to social desirability bias, fear of negative consequences, and lack of anonymity.
• Participants want to please the researcher or the organization, so they may not provide honest feedback.
• The same effect is magnified when there is a power imbalance between the researcher and the participant.

2. What are some alternative ways to gather user feedback that are less prone to bias?

• Gather quantitative data through task success rates, durations, and anonymous user satisfaction surveys or third-party tools.
• Use open-ended questions to understand user behaviors and motivations, rather than directly asking about likes or dislikes.
• Combine qualitative and quantitative research methods to get a more comprehensive and reliable understanding of user needs and preferences.

[02] Addressing Stakeholder Demands

1. Why do researchers sometimes still ask "Do you like it?" questions, even though they know it's problematic?

• Stakeholders often demand to know if people will like a product or feature before investing in it, and researchers feel pressure to provide this information.

2. How can researchers address stakeholder demands in a more constructive way?

• Encourage stakeholders to articulate their underlying needs more clearly, such as understanding how many people are likely to use or buy the new design.
• Explain that liking a product does not necessarily translate to usage or purchase, and that research should focus on understanding user needs and behaviors.
• Propose a well-planned combination of quantitative and qualitative research activities to answer the stakeholders' questions in a more reliable and meaningful way.