Tese
Metodologia de aprendizado AutoML baseado em informações de complexidade de instâncias
Fecha
2020-12-07Autor
Habib Asseiss Neto
Institución
Resumen
Analysis of biological data is a very important task since it allows one to obtain useful
information and to expand knowledge about some biological domain. Biological materials can be analyzed by several techniques and a widely used method is the infrared
spectroscopy analysis, which allows the information extraction through the emission of
infrared light in the samples. Spectroscopy techniques produce a large volume of data, which make manual analysis by experts complex. Computer Science, through Machine Learning, can help in this task, o↵ering ways to understand and produce important knowledge from spectral samples. Convolutional Neural Networks, generally successfully applied to image
recognition, are specifically suitable for spectral data of samples, since spectral structure can be seen as an image. This work conducts experiments in order to detect the occurrence of adulteration in bovine milk through infrared spectroscopy analyzes using a proposed convolutiona neural network architecture and ensemble decision trees models. In the experiments, spectral data from thousands of pure and adulterated samples of bovine milk were subjected to the proposed convolutional neural network, to the ensemblee models and
to other commonly used methods for this purpose, allowing a comparison of di↵erent
approaches. The proposed approach was able to detect adulterants with an accuracy of
up to 98.76% for convolutional neural network and tree embedding, while the commonly
used baseline methods produced average accuracy of 65.88%. Despite the good performance of the convolutional neural network for the milk problem, designing neural network architectures that o↵er good performance for generic problems is a challenging task. Generally, the search for an adequate architecture is a specific process for the problem addressed and it is conducted by specialized scientists through manual and extensive tests, and also prior knowledge on similar problems is required. Automated Machine Learning, or AutoML, can collaborate in this process, since its objectives is the search for the most adequate architectures for the input problem in an automated way and no human interaction.
However, Machine Learning methods, or even AutoML methods, generally do not consider individual characteristics from instances that belong to the analyzed datasets. By considering information that reflects each sample’s characteristics, it is possible to evaluate the methods’ habilities. Item Response Theory (IRT) is a psychometrics approach that can be adapted to Machine Learning and can o↵er complexity descriptions on an instance level, and also characterize inherit abilities to Machine Learning models. In this work, we propose an innovative methodology based on AutoML and IRT that is capable of o↵ering high explicability of Convolutional Neural Network models. The method selects the most adequate instances to be submitted to di↵erent models, based on the diculty and discrimination information and also models abilities. Several experiments were conducted in order to evaluate the methodology viability on real datasets, comparing the results with other benchmark methods. Results show that the proposed method presents, in most cases, better performance over the other methods.
The proposed methodology was capable of generating average accuracies of 96.96% for
a specific dataset, while a Convolutional Neural Network manually created presented
78.43%, a voting approach with hundreds of models presented 69.06% and an already
existing AutoML system presented 91.81% of accuracy.