Discuss the effectiveness and limitations of a chat bot in conducting surveys.

Your critique should contain the following elements:

1. Provide a complete citation of the article (Title, authors, journal, volume, year and pages).

2. What are the research questions/hypotheses?

3. What are the major theories that it examines (be sure to briefly explain the theory)?

4. What is the research methodology (sample size, independent variables, dependent variables and how were they measured)?

5. Summarize the major results

6. What does the author say is the major contribution of the study?

7. What would you say are the strengths and weaknesses of the study with a focus on its methodology?

 

Tell Me About Yourself: Using an AI-Powered Chatbot to
Conduct Conversational Surveys with Open-ended
Questions
ZIANG XIAO, University of Illinois at Urbana-Champaign, USA
MICHELLE X. ZHOU, Juji. Inc., USA
Q. VERA LIAO, IBM Research AI, USA
GLORIA MARK, University of California, Irvine, USA
CHANGYAN CHI, Juji. Inc., USA
WENXI CHEN, Juji. Inc., USA
HUAHAI YANG, Juji. Inc., USA
The rise of increasingly more powerful chatbots offers a new way to collect information through conversational
surveys, where a chatbot asks open-ended questions, interprets a user’s free-text responses, and probes answers
whenever needed. To investigate the effectiveness and limitations of such a chatbot in conducting surveys, we
conducted a field study involving about 600 participants. In this study with mostly open-ended questions, half
of the participants took a typical online survey on Qualtrics and the other half interacted with an AI-powered
chatbot to complete a conversational survey. Our detailed analysis of over 5200 free-text responses revealed
that the chatbot drove a significantly higher level of participant engagement and elicited significantly better
quality responses measured by Gricean Maxims in terms of their informativeness, relevance, specificity, and
clarity. Based on our results, we discuss design implications for creating AI-powered chatbots to conduct
effective surveys and beyond