LEGO© Mindstorms NXT and Q-Learning: a teaching approach for robotics in engineering

Abstract

Robotics has become a common subject in many engineering degrees and postgraduate programs. Although at undergraduate levels the students are introduced to basic theoretical concepts and tools, at postgraduate courses more complex topics have to be covered. One of those advanced subjects is Cognitive Robotics, which covers aspects like automatic symbolic reasoning, decision-making, task planning or machine learning. In particular, Reinforcement Learning (RL) is a machine learning and decision-making methodology that does not require a model of the environment where the robot operates, overcoming this limitation by making observations. In order to get the greatest educational benefit, RL theory should be complemented with some hands-on RL task that uses a real robot, so students get a complete vision of the learning problem, as well as of the issues that arise when dealing with a physical robotic platform. There are several RL techniques that can be studied in such a subject; we have chosen Q-learning, since is a simple, effective and well-known RL algorithm.

In this paper we present a minimalist implementation of the Q-learning method for a Lego Mindstorms NXT mobile robot, focused on simplicity and applicability, and flexible enough to be adapted to several tasks. Starting from a simple wandering problem, we first design an off-line model of the learning process in which the Q-learning parameters are studied. After that, we implement the algorithm on the robot, gradually enlarging the number of states-actions of the problem. The final result of this work is a teaching framework for developing practical activities regarding Q-learning in our Robotics subjects, which will improve our teaching.