HomeAnalysisIIIT-H Sign-Language Plugin Targets a Major Digital Access Gap

IIIT-H Sign-Language Plugin Targets a Major Digital Access Gap

A browser plugin developed by two students at the International Institute of Information Technology, Hyderabad (IIIT-H), is designed to make online content more accessible to people who use Indian Sign Language (ISL).

The tool, called SignPlug, allows users to select text on a webpage and click a “Click To Sign” button. The selected content is then presented through a 3D animated avatar using ISL. The project was developed by third-year BTech students Masthan Sharif Shaik and Mohammed Raza at IIIT-H’s Centre for Visual Information Technology.

According to a report by Deccan Chronicle, tests involving deaf and hard-of-hearing signers found that users understood about 89 per cent of the sentences presented by the avatar. The students were supervised by C.V. Jawahar and Vinay Namboodiri of the University of Bath in the UK.

The project is intended to address a language barrier in digital access. The researchers said that, for many people with hearing disabilities, sign language is their first language, while English, Hindi and Telugu may be acquired as additional languages. ISL also has its own grammar and word order, meaning that directly reading conventional written text can be difficult for some users.

To address this, the system converts English sentence structures into ISL word order before generating the signing sequence. The report gives the example of “He will come home” being rendered as “He home come”. Words without matching signs are spelt out letter by letter.

The researchers used videos from the official Indian Sign Language Research and Training Centre dictionary to reconstruct signers’ body, hand and finger movements in 3D. They also developed an annotation tool called Gizmo to correct key poses where automated reconstruction did not accurately capture finger movements.

This detail is significant because small errors in hand or finger positions can change the meaning of a sign. Instead of manually correcting every frame, the tool allows the researchers to adjust selected key poses before combining the stored movements into a continuous signing sequence.

The plugin is designed to run in a regular web browser without specialised motion-capture equipment, a graphics processing unit or a server-based artificial-intelligence model. That could reduce the hardware and infrastructure requirements for users, although the report does not establish how the system would perform across a broad range of websites, devices or internet conditions.

The current version has a recognised limitation: it does not yet reproduce facial expressions. Facial movements convey grammatical information in ISL, and their absence can affect the completeness of a signed translation. Shaik said the team plans to incorporate facial expressions in a future iteration.

Jawahar said ISL continued to lack adequate resources and technology. The research group is working on both sign-language synthesis, which converts text into signs, and sign-language understanding, which interprets signs.

The research paper, “SignPlug – ISL Plugin for the Web”, has been accepted for presentation at the 17th Indian Conference on Computer Vision, Graphics and Image Processing, scheduled to be held in Kolkata in December. The reported test result indicates a promising prototype, but wider deployment would depend on improving facial expression, expanding the sign vocabulary and assessing how reliably the system handles varied online language and content.


RELATED ARTICLES

Most Popular

Latest News