Handspeaker: : Animation of the Mexican Sing Language Dactylology

Hand Speaker: Animation of the Mexican Sing Language Dactylology

G., JEREZ-GOMEZ*

Escuela de Ingeniería, Universidad Modelo; 15233939@modelo.edu.mx

A.C., CASTRO-CANCHOLA

Escuela de Ingeniería, Universidad Modelo; 15234419@modelo.edu.mx

P.I., ROSAS-PEREZ

Escuela de Ingeniería, Universidad Modelo; 19198549@modelo.edu.mx

R., NAVA-MARTINEZ

Escuela de Ingeniería, Universidad Modelo; rutilio.navam@modelo.edu.mx

The Mexican Sign Language Fingerspell Translator represents a critical tool for individuals attempting to learn Mexican Sign Language (LSM). This innovative tool offers a 3D visualization of each letter in LSM dactylology. Although assistive technologies such as hearing aids and closed captioning have significantly enhanced communication capabilities, a substantial communication gap persists between sign language users and non-users. To build up and provide a more interactive and affordable learning method for understanding and assimilate abstract concepts in LSM. This system has been developed to translate written text into Mexican Sign Language dactylology, moreover the system was created using Unity, a real-time 3D development platform, to facilitate real-time translation and visualization.

CCS CONCEPTS •

Software and its engineering

Medium Relevance

• Software creation and management
• Designing software
• Software design engineering

Additional Keywords and Phrases: Dactylology, Mexican Sign Language, Animation

1. Introduction

In Mexico, approximately 2.3 million people suffer from hearing disabilities [1]. Most of these individuals do not have access to adequate interventions, and when they do, the processes can take years and incur significant amount of money. This has an impact on the lives of affected individuals, as their limitations prevent them from having the same opportunities as others.[pic 1]

According to Guerrero-Balaguera & Pérez-Holguín [2], most people with hearing impairments interact with each other using sign language; otherwise, they often need an interpreter to communicate with those who do not understand their language. This situation creates a communication barrier and restricts the daily lives of these individuals. Marzo Peña, L. A., Rodríguez Fleitas, D. C., and Fresquet Pedroso, D. C. [3] state, “In the education of deaf people, there has been a historical debate regarding the methods, trends, and models used for their educational process. Some advocate for more hierarchical positions, while others support respect for their linguistic, communicative, and cultural diversity, such as sign language.

“Considering the context of hearing disabilities in Mexico, we offer a project involving technology to gap opportunities where communication is not a hindrance for either the receiver or the transmitter. Rojas Hernández, R., González Jaime, E.I., Trujillo Mora, V., López Chau, A., & González Morán et al. [4] mention that “The teaching of Mexican Sign Language (LSM) can be done through very short videos, as the positions and movements of the hand to represent a letter or word do not require much time.” Many users have taken it upon themselves to share videos showcasing the available vocabulary in LSM so that people can learn on their own without the need for an instructor. Utilizing technological tools, which are advancing rapidly, to develop support systems for people with disabilities is becoming increasingly common, as computers are interacting more and more with people and their daily lives.

According to Pathan RK, Biswas M, Yasmin S, Khandaker MU, Salman M. et al [5], many techniques have been developed using sensor data processing and motion detection through the application of various algorithms and dynamic methods such as machine learning and deep learning. Depending on the methodologies, researchers have proposed their ways of classifying sign languages, and based on this, the project can be developed. The overarching goal of this project is to design a translator from written text to the Mexican Sign Language alphabet to promote the inclusion of people with hearing disabilities.

The proposed design is validated only for the Mexican Sign Language alphabet, where only spelling will be shown. According to Cruz-Aldrete, M., DIELSEME [6] constitutes a valuable support resource to facilitate communication with people with hearing disabilities.

1. MATERIALS AND TOOLS

1. Mexican Sign Language Dactylology

According to Marzo Peña, Alejandro, et al. [3], "Sign languages are visual-gestural-dimensional languages based on the use of hands, eyes, face, mouth, and body. They are languages meant to be seen, not heard, and they fulfill the same functions as spoken languages for hearing people." These languages possess their own grammar, syntax, and lexicon, enabling people with hearing impairments in Mexico to express and communicate with each other. However, a significant percentage of the population is unfamiliar with it, complicating their interaction with the public.

Papatsimoul, M., Sarigiannidi, P., & F. Fragulis, G. [7] mentions that by incorporating sign language interpretation into such projects, we promote inclusion, providing equal access to information and entertainment while strengthening community bonds. Mexican Sign Language dactylology consists of both static and dynamic signs, the latter being those that exhibit certain movements indicated by a black arrow (see Fig 1).

[pic 2]

Fig 1. Mexican Sign Language Dactylology [9]. Photograph by Conadis. [Public domain], via. Gobierno de México. (www.gob.mx)

1. Unity 2022.3

Unity (game engine) was employed as the main tool since this software facilitates the development of 3D animations. Its graphics engine allows for the creation of realistic three-dimensional environments, which aids in visualizing data such as the animation of Mexican Sign Language dactylology. Additionally, its multiplatform support ensures that simulations can be run on a variety of devices, from desktop computers to mobile devices, making it an ideal platform for the project's development.

1. METHODOLOGY

Using Unity, the hand animation was created through a default template from the UNITY Assets. This 3D template covered the wrist to the fingertips (see Fig 2).

...