The simultaneous-sequential nature of sign language production, which employs hand gestures and body motions combined with facial expressions, still challenges sign language recognition algorithms. This paper presents a method to recognize Brazilian Sign Language (Libras) using Kinect. Skeleton information is used to segment sign gestures from a continuous stream, while depth information is used to provide distinctive features. The method was assessed in a new data-set of 107 medical signs selected from common dialogues in health-care centers. The dynamic time warping-nearest neighbor (DTW-kNN) classifier using the leave-one-out cross-validation strategy reported outstanding results.991439140214th European Conference on Computer Vision (ECCV)OCT 08-16, 2016Amsterdam, NETHERLANDS