Uso de redes neurais convolucionais para classificação de sinais da linguagem brasileira de sinais aplicados ao ensino de línguas

Abstract

The present work shows the use of convolutional neural networks applied to detection and classification of continuous signs from Brazilian Sign Language (LIBRAS) in videos captured by conventional computer cameras, and its use in a methodological platform for teaching LIBRAS. The goal of this work is to propose a processing methodology that allows the classification of signs performed by people of different genders, physical frames and skin colors, suffering the lowest interference as possible from video background and using common computer cameras, in order to have a methodology accessible to several future applications. Also, this project experiments the viability of the use of this methodology in the field of basic education, providing accessibility and inclusion to LIBRAS teaching. To do this, the Mediapipe algorithm will be used to extract data from videos, the FastDTW algorithm will be used to standardize this data, neural networks based on the TensorFlow and Keras libraries and game development structured on Pygame. All these tools will work with the Python programming language to produce a neural network trained over several databases to recognize and differentiate the LIBRAS signs.