In this document we describe a hand gesture classification system of the Colombian Sign Language for both dynamic and static signs, based on Computer Vision and Machine learning. The proposed processes sequence is divided in four stages: acquisition of RGB-D image, extraction of the blob closest to the sensor, detection and validation of the hand, and classification of the sign entered. The results obtained are for multi-class classifiers with a self-captured dataset of 3.600 samples. As a conclusion we found that the best choice for descriptor-classifier according to sign type are HOG-SVM for static signs with an accuracy of 98%, and SVM classifier besides the trajectory-based descriptor with an accuracy of 94%.