A new paper in the Quarterly Journal of Economics, published by Oxford University Press, shows customer service workers using artificial intelligence assistance become more productive and work faster. The effects vary significantly, however. Less experienced and lower-skilled employees improve both the speed and quality of their work, while the most experienced and highest-skilled workers see small gains in speed and small declines in quality. The researchers also found that AI assistance can help worker learning and improve English fluency, particularly for international workers.
Computers and software have transformed the economy with their ability to perform certain tasks with far more precision, speed, and consistency than humans. Yet, despite significant advances in traditional computing, some workplace activities—writing emails, analyzing data, or creating presentations—are difficult to codify and have defied computerization.
Computers historically did well with pre-programmed instructions, making them effective for tasks with explicit rules. Computerization has therefore disproportionately decreased the demand for workers performing routine and repetitive tasks such as data entry, bookkeeping, and assembly line work, reducing wages in these jobs. But computerization has increased the demand for workers who possess complementary skills such as programming, data analysis, and research. As a result, technology-related shifts in the labor market have contributed to increased wage inequality.
Researchers here studied the impact of generative AI on productivity and workers' experience in the customer service sector by examining the use of a new chat assistant by 5,172 customer support agents. The agents worked at a Fortune 500 firm that sells business process software.
The investigation had several findings. First, AI assistance increased worker productivity, resulting in a 15 percent increase in the number of chats that an agent successfully resolved per hour.
Second, the impact of AI assistance varies widely. Less-skilled and less-experienced workers improve significantly across all productivity measures, including a 30 percent increase in the number of issues resolved per hour. The AI tool also helps newer agents become skilled agents more quickly AI-assisted agents with two months of tenure performed just as well as regular agents with more than six months of tenure. In contrast, AI has little impact on the productivity of higher-skilled or more experienced workers. Indeed, the investigation here showed that AI assistance leads to a small decrease in the quality of conversations conducted by the most skilled agents.
Third, agents who follow AI recommendations closely generally see larger gains in productivity, and adherence rates increase over time. The investigators here also analyzed the text of agents' chats and showed that access to AI improves their English language fluency, especially among international agents.
Finally, using AI appears to improve customer behavior. Contact center work is challenging, and employees face hostile interactions from anonymous, frustrated customers frequently. The researchers here show that access to AI assistance significantly improves the treatment of agents, as reflected in the tone of customer messages. Customers are also less likely to question agents' competence or ask to speak to a supervisor.
The paper, "Generative AI at Work," is available (at midnight on February 4th) at https://academic.oup.com/qje/article-lookup/doi/qjae044/qje/qjae044 .
Direct correspondence to:
Erik Brynjolfsson
Jerry Yang and Akiko Yamazaki Professor and Director of the Stanford Digital Economy Lab
Stanford Institute for Human-Centered Artificial Intelligence
353 Jane Stanford Way
Stanford University
Stanford, CA 94305
