Using AI for research

🌈 Abstract

The article explores the perspectives of UX professionals on using large language models (LLMs) for user experience research. It investigates the confidence and skepticism of UX practitioners towards outsourcing research processes to AI-powered chatbots.

🙋 Q&A

[01] Using AI for research

1. How confident are non-research UX practitioners in outsourcing UXR processes to LLMs?

• UX practitioners with more experience in UX research exhibited lower confidence in outsourcing critical UXR processes to LLMs, due to their deeper understanding of the nuances and complexities involved in qualitative research.
• Less experienced practitioners showed higher confidence levels, as they viewed LLMs as a valuable tool that could bridge gaps in their skill set and streamline their workflow.

2. What UXR processes are most likely to be outsourced to LLMs?

• Tasks such as initial methodology brainstorming and screener survey creation were frequently cited as ideal for outsourcing to LLMs, as they are often viewed as more straightforward, repetitive, and requiring input from additional team members.
• Critical tasks like quantitative data analysis, test participant definition, creation of artifacts, and data synthesis were less likely to be entrusted to LLMs, as practitioners were reluctant to relay their responsibility for human empathy, contextual understanding, and interpretative skills to a machine.

3. What are the overall sentiments of UX professionals towards using AI chatbots for UX research?

• Respondents articulated a range of perspectives, with the majority falling into three categories: reflecting the benefits associated with it (e.g., reducing bias), how they would use it (e.g., to help with data analysis), and concerns they have for the technology (e.g., privacy).
• Those with higher average satisfaction rates for AI chatbots tended to have more positive comments, noting the benefits of the AI-powered chatbots and how they would use the technology.

[02] AI for UX survey

1. What were the key findings from the survey conducted as part of the study?

• The survey involved 16 UX professionals from various subfields, including design, content strategy, and management, as well as adjacent disciplines.
• Most participants rated themselves moderately to extremely familiar with user research.
• The survey aimed to gauge participants' levels of expertise, their satisfaction with popular AI-powered chatbots, their assessment of the importance of various research processes, their likelihood of using AI chatbots for different UXR tasks, and their overall sentiment towards these tools.