info:eu-repo/semantics/article
A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning
Fecha
2020-05-19Registro en:
Barsce, Juan Cruz; Palombarini, Jorge Andrés; Martínez, Ernesto Carlos; A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning; Sociedad Argentina de Informática E Investigación Operativa; SADIO Electronic Journal of Informatic and Operation Research; 19; 2; 19-5-2020; 2-27
1514-6774
CONICET Digital
CONICET
Autor
Barsce, Juan Cruz
Palombarini, Jorge Andrés
Martínez, Ernesto Carlos
Resumen
Optimization of hyper-parameters in real-world applications of reinforcement learning (RL) is a key issue, because their settings determine how fast the agent will learn its policy by interacting with its environment due to the information content of data gathered. In this work, an approach that uses Bayesian optimization to perform an autonomous two-tier optimization of both representation decisions and algorithm hyper-parameters is proposed: first, categorical / structural RL hyper-parameters are taken as binary variables and optimized with an acquisition function tailored for such type of variables. Then, at a lower level of abstraction, solution-level hyper-parameters are optimized by resorting to the expected improvement acquisition function, whereas the categorical hyper-parameters found in the optimization at the upperlevelof abstraction are fixed. This two-tier approach is validated with a tabular and neural network setting of the value function, in a classic simulated control task. Results obtained are promising and open the way for more user-independent applications of reinforcement learning.