Classificação de sinais dinâmicos de Libras relacionados ao contexto hospitalar utilizando redes neurais recorrentes
Loading...
Date
Journal Title
Journal ISSN
Volume Title
Publisher
Universidade Federal de Juiz de Fora (UFJF)
DOI
Abstract
Limited communication between healthcare professionals and deaf patients still
represents a significant obstacle to ensuring inclusive and equitable care, especially in
clinical contexts in which the speed and accuracy of information are essential. In this
scenario, this work proposes a method for the classification of ten dynamic signs of
Brazilian Sign Language (Libras) related to the healthcare domain, encompassing the
construction of a dataset as well as the development and evaluation of machine learning
models. The constructed dataset involved the participation of professionals fluent in
Libras, ensuring linguistic quality and diversity of execution. A total of 492 videos were
recorded, corresponding to ten signs commonly used in hospital contexts, performed by
nine individuals under controlled conditions, with variations in clothing, appearance, and
gesture repetition in order to increase sample variability. The preprocessing stage included
frame selection through probabilistic sampling and the extraction of three-dimensional
hand and body pose landmarks, which correspond to anatomical reference points that
describe the configuration and movement of the body over time. These landmarks were
organized into temporal sequences, preserving the dynamics of the gestures. Based on this
data, different classification approaches were investigated using neural network models with
Long Short-Term Memory (LSTM) layers, capable of modeling the temporal dependencies
present in the sequences of sign movements. Initially, the problem was formulated with ten
classes. The analysis of confusion matrices revealed specific confusions between pairs of
signs, which motivated the adoption of a hierarchical strategy based on specialist models,
in which the problem was reorganized into seven classes, complemented by classifiers
dedicated to the conflicting pairs. This strategy resulted in significant improvements in
accuracy and result stability, achieving an average accuracy of 92.44% in the experiments.
Evaluation on an additional test dataset confirmed the system's good generalization
capability, with an average accuracy of 98.92% and a low standard deviation. The results
indicate that the combination of a well-structured dataset, appropriate preprocessing
techniques, and a hierarchical architecture of specialist models constitutes a promising
solution for Libras sign recognition, contributing to the reduction of communication
barriers and with potential application in healthcare communication support systems.
Description
Citation
Collections
Endorsement
Review
Supplemented By
Referenced By
Creative Commons license
Except where otherwise noted, this item's license is described as Acesso Aberto

