Colombian sign language analysis and recognition

Abstract

We present a methodology for the analysis and recognition of signs on the first open database of the Colombian sign language, CSL50: Colombian Sign Language Video and Inertial Measurement Dataset. In particular, the proposed methodology has a focus on motion energy analysis from a multimodal perspective; video sequences and inertial measurements. And additionally, we present the use of the vision transformer model for the classification of CSL signs, obtaining a top 1 accuracy of around 0.7 and a top 3 accuracy greater than 0.9. This study presents the bases to develop more technological tools that allow a better understanding of the colombian sign language.