The Fairwork Project Takes a Critical Look at Crowdsourcing Platforms

The Fairwork Project has published its Cloudwork Ratings 2023 report evaluating the labor practices of 15 platforms, among them high-profile names including Upwork, Fiverr, and data-for-AI leader Appen. The project, now in its third year, is a collaboration between the Oxford Internet Institute and researchers at the Berlin Social Science Center (the WZB).

Lead researcher Jonas C. L. Valente commented in a July 20, 2023 press release, “Many people rush to the latest technologies and enjoy the fruits of fast-paced innovation without stopping to think about how the product arrived to where it got.”

The Fairwork Project evaluates digital labor platforms according to five categories of fair labor practices: pay; conditions; contracts; management; and representation. Platforms can earn up to two points per category for a maximum score of 10 points.

This year, none of the 15 platforms reviewed earned 10 points, representing compliance with minimal standards for fair labor practices. The highest score, netted by ComeUp, Prolific, and Terawork, was 5/10.

Appen, with 3/10, performed better than 10 other platforms. Four companies, including Amazon Mechanical Turk, which researchers have used to crowdsource tasks related to machine translation (MT), were awarded zero points.

Platforms Upwork and Fiverr — rated 3/10 and 1/10, respectively — also advertise freelance translation services on their landing pages. 

Although the 2023 report spans industries, past reports have centered on specific fields, notably the Fairwork 2022 Translation & Transcription Platform Ratings. Translated and TranscribeMe topped the list of nine platforms, with scores of 8/10 and 7/10 apiece, but researchers could not confirm that the bottom-ranked four platforms (0/10) met any minimum standards of fair work.

In response to recommendations from the Fairwork Project, Appen reportedly modified its policies to simplify the appeal process for workers, made it easier for workers to present requests to the platform, and reconfigured its system for improved management of job availability.