A Research Collab to Use Non-Mediated Sign Language for AI Training

A GBP 3.5m (USD 4.7m) initiative will combine leading linguistics and language technology experts from the UK and Japan on a research project aiming to steer AI sign language training toward the fluid, natural conversations occurring within the deaf community, according to a press release from January 21, 2026.

“Most AI research on sign language has used video of interpreters signing to cameras. We know that’s not how Deaf people naturally communicate,” commented Professor Richard Bowden of the University of Surrey. 

Jointly funded by UK Research and Innovation (UKRI) and the Japan Science and Technology Agency (JST), the project is known as “Understanding Multilingual Communication Spaces” (UMCS) and prioritizes authentic conversational data, analyzing the nuances of how deaf individuals interact — such as how they switch speakers, give feedback, and maintain visual contact.

The project is expected to design human-centric AI and augmented reality (AR) tools to provide instant translation between British Sign Language (BSL), Japanese Sign Language (JSL), English, and Japanese. 

“What excites me about this project is that we’re working with authentic conversations between Deaf signers. That will give us much richer insight into how people really interact – and help us build AI systems that reflect that complexity,” commented Bowden. 

Joining him in this initiative is an interdisciplinary team bringing together sociolinguists, deaf studies experts, and AI engineers from three UK institutions (University of Surrey, Heriot-Watt University, and University College London) and four Japanese academic centers (National Institute of Informatics, Tsukuba University of Technology, University of Tokyo, and Kyoto University).

Industrial partners, such as the Surrey-based startup Signapse Ltd (co-founded by Bowden) and Japan’s NHK Enterprises, will help transition the research from the lab to the real world.

Inclusive Language AI

Professor Mayumi Bono, from the National Institute of Informatics in Japan, remarked about the new project that “today’s AI systems demand large-scale, text-linked data. As the field moves from ‘corpus to dataset,’ researchers are calling for an inclusive science that bridges linguistics and AI while centring on the lived realities and linguistic intuitions of deaf signers.” 

The study is slated to last five years and will focus on capturing the complex dynamics of natural dialogue, including turn-taking, backchanneling (the subtle nods or signs that show a listener is following), and how signers correct misunderstandings in real-time. 

This is the second major AI sign language project launched within a year involving Professor Bowden: The UK’s Engineering & Physical Sciences Research Council (EPSRC) awarded a GBP 8.45m (USD 10.5m) grant in January 2025 to a consortium of universities to develop SignGPT, a foundational large language model, with Bowden’s direct participation.

SignGPT focuses on the fundamental grammatical challenges of sign language, with the goal of creating a scalable framework that can eventually be applied to various sign languages globally. Although currently focusing on British Sign Language, researchers believe that solving the core translation hurdles for one language will indeed pave the way for universal accessibility via AI.