What Microsoft’s Misunderstood Copilot Study Actually Means for the Language Industry

Slator’s Head of Research Anna Wyndham joins Florian on the pod to discuss Microsoft’s research paper “Working with AI: Measuring the Occupational Implications of Generative AI”, a study that stirred significant debate across social media.

The paper, based on 200,000 anonymized Microsoft Copilot interactions, aims to understand what tasks people ask AI to perform and how effectively those tasks are completed. Pairing this with the US O*NET database of occupational tasks, researchers created an “AI applicability score” to assess overlap between AI-capable tasks and real-world job functions.

Anna emphasizes that the researchers distinguish between AI performing individual tasks and full jobs. Even the most affected roles, like interpreters and translators, show only partial overlap, around 50%, with activities AI can complete.

Florian and Anna stress that the research does not claim AI will replace top-ranked occupations. Rather, it shows where AI is most often helpful, with knowledge-based activities like writing, summarizing, and gathering information topping the list. 

Subscribe on YouTube, Apple Podcasts, Spotify, and elsewhere

The Microsoft researchers also acknowledge key limitations. For example, jobs are more than bundles of disconnected tasks; they involve context, judgment, and synthesis, often referred to as the “glue” that AI lacks. Additionally, Anna points out that Copilot’s integration into tools used by knowledge workers may bias the results in its favor.

Ultimately, the duo agree the paper validates what’s already known: AI is helpful for language-related tasks, but not transformational enough yet to supplant the people who perform them.