Tesis
QFCS: A Fuzzy LCS in Continuous Multi-Step Environments with Continuous Vector Actions
Autor
Ramírez Ruiz, José A.
Institución
Resumen
This document presents a doctoral dissertation which is a requirement for the Ph.D. degree in Information Technologies and Communications from Instituto Tecnológico y de Estudios Superiores de Monterrey (ITESM), Campus Monterrey, major in Intelligent Systems in the field of Learning Classifier Systems (LCS). The dissertation introduces a new LCS, called QFCS, that is able to deal with problems defined over continuous variables. These problems are important because real life is modeled in that way. LCSs are systems with a set of rules that compete and that can learn from and adapt to the environment. These properties are very desirable in intelligent systems because they allow the systems to adjust to subtle details. Traditionally, in Artificial Intelligence, designers have to pre-adjust the parameters. This made the developer not to take into account those subtle details and, consequently, deal with them during ex- perimentation. LCSs make use of reinforcement learning and of evolutionary computing to deal with the proper adjustment of those subtleties. But, LCSs in their beginnings have been designed to solve problems that can be defined in a discrete form. Lately, researchers have tried to extend the approach to deal with problems in the continuum. This task has shown to be far away of being solved. Thus, there have been many ap- proaches to adapt these systems to continuous variables. One of them has been the introduction of Fuzzy Logic to model the continuous environment. In this way, little by little LCSs have been extended to tackle more problems in the continuum, increasing, little by little, their related difficulty. Some of the problems solved with this approaches are the learning of continuous functions, the frog problem and navigation tasks. Learning of continuous functions is a problem where some continuous input enters to the system and the corresponding output is obtained, but the system does not know what this output is, all the system knows is the amount of reward it receives for each output made. This problem is of one-step since the system has to place an output once. The frog problem consists of a frog that lives with a fly in a line. The frog has to jump once and catch the fly. Since the frog lives in a line, the environment is continuous. The length the frog jumps is also continuous. This is one-step since the frog jumps once. The frog receives a reward at each time it jumps even if it does not trap the fly. Navigation tasks are more complex problems since they are multi-step. This means the system has to act more than one time to reach the goal. In this case, it moves many times to reach another place. The environment can be discrete but it is more complex if it is continuous. The actions are a set of discrete vectors but, in a more complex form, they could be continuous. The reward is given when the system reaches the goal. ix ?The complexity for LCSs with the problems described before is given by the set of continuous actions, because rules of LCSs relate the states of the problem with one action. Thus, to model continuous outputs would required a set of infinite number of rules. This is impossible because rules are countable and the continuum is not. QFCS introduces fuzzy systems in the rules to model relationships of the form: many states to many actions. This is a novelty in LCSs literature where only single fuzzy rules have been used. QFCS uses a matrix to learn a prediction of the payoff to be obtained per each fuzzy system. This is also a novelty because traditional LCSs use one value to predict the obtained reward. In this way, QFCS was designed following to different approaches, one that has fixed fuzzy sets and the other with unfixed fuzzy sets as inputs to the fuzzy systems. The second approach is a generalization and it is proposed to eliminate the restrictions imposed by the use of fixed fuzzy sets. This QFCS was tested with the frog problem to compare it with the literature and with five more different problems that introduced different levels of complexity. Three of them were about performing navigation tasks in one and two dimension spaces with obstacles. The last two dealt with an inertial particle. The navigation tasks were in continuous spaces with a set of continuous vector actions. A reward was given in those regions the system had to reach. The particle problems were about moving a particle from one position to another one. These problems were the more complex ones because of the inertia of the particle. Results showed that QFCS is capable of solving these kind of continuousproblems and that there is not too much difference in performance between the two approaches of QFCSs. Liberating the fixed fuzzy sets did not represent an advantage.