How AI Is Changing the Labor Market: An Analysis of Bing Copilot Data

Today, generative artificial intelligence (AI) is becoming part of everyday life and work. This article focuses on a study that analyzes 200,000 anonymized conversations between users and the Microsoft Bing Copilot system. These data were collected in the United States over nine months in 2024. The study, authored by Kiran Tomlinson, Sonia Jaffe, Will Wang, Scott Counts, and Siddharth Suri, examines how AI assists with work activities and the implications for various occupations. The authors use the O*NET database, which breaks occupations down into hierarchical work activities, and combine it with wage and employment data from the Bureau of Labor Statistics (BLS).

A key element of the analysis is the division into two categories: user goals, where AI helps with tasks that a person wants to accomplish, and AI actions, where AI itself performs activities. For example, if a user seeks advice on how to print a document, the user goal is to operate office equipment, while the AI action is to train others in using the equipment. These data show that AI is most commonly used for gathering information, writing, and communication, which are typical of knowledge work.

Methodology: How the Data Were Processed

The study uses two datasets: Copilot-Uniform, containing 100,000 randomly selected conversations to provide a representative view, and Copilot-Thumbs, containing another 100,000 conversations in which users provided feedback using thumbs up or thumbs down. The conversations were classified into intermediate work activities (IWAs) from O*NET, comprising 332 general activities such as "gather information from physical or electronic sources."

Classification was performed using the GPT-4o model in a two-stage process: first, the goals and actions were summarized, and then binary classification was used to determine whether a conversation corresponded to a given IWA. Success was measured by user feedback, task completion rate, and impact scope, which assesses how much of the activity AI covers. Based on these measures, an AI applicability score was calculated for each occupation, taking coverage, success, and scope into account.

This approach enables relative comparisons between occupations rather than absolute percentages because the thresholds for "coverage" affect the results. For example, if an activity appears in at least 0.05% of conversations, it is considered covered.

Results: Most Common Activities and Occupations at Risk

The data show that the most common user goals include gathering information (such as "gather information" or "obtain information about goods or services"), writing (such as "write materials for artistic or commercial purposes"), and communication (such as "provide information to clients"). Common AI actions include providing assistance, explaining, and advising, such as "provide general advice" or "explain technical details."

Success rates are high for writing and research, where the share of positive feedback exceeds 50%, while data analysis and visual design have lower success rates. The scope of impact is greater when assisting users than when AI takes direct action. For example, 40% of conversations involve different sets of activities for users and AI, indicating an asymmetry.

The most affected occupations have high AI applicability scores, indicating potential for automation or enhancement. Below is a table of the 10 occupations most at risk according to the score (the average of user goals and AI actions). The data are based on Table 3 of the study, where coverage is the share of covered activities, completion is the completion rate, scope is the extent of impact, and score is the overall score. Employment figures are for the United States.

These occupations have high scores due to their overlap with activities such as writing, information gathering, and communication. Conversely, occupations involving physical labor, such as dishwashers or roofers, have the lowest scores.

Discussion: What This Means for the Future

The study emphasizes that AI primarily assists with knowledge work, but this does not automatically mean job losses. For example, after ATMs were introduced, bank tellers shifted their focus to client relationships, which led to more branches. The authors note several limitations: the data come from only one system, O*NET may not be up to date, and conversations may not reflect all work activities. Future research should monitor how occupations change and how new ones emerge.

A comparison with predictions from the work of Tyna Eloundou, Sam Manning, Pamela Mishkin, and Daniel Rock shows a correlation of 0.73 at the occupational level, confirming that actual AI use aligns with expectations. However, the impact on wages and employment remains an open question—AI may increase productivity while also changing the structure of work.