Efficient concept formation in large state spaces
Paper in proceeding, 2018

General autonomous agents must be able to operate in previously unseen worlds with large state spaces. To operate successfully in such worlds, the agents must maintain their own models of the environment, based on concept sets that are several orders of magnitude smaller. For adaptive agents, those concept sets cannot be fixed, but must adapt continuously to new situations. This, in turn, requires mechanisms for forming and preserving those concepts that are critical to successful decision-making, while removing others. In this paper we compare four general algorithms for learning and decision-making: (i) standard Q-learning, (ii) deep Q-learning, (iii) single-agent local Q-learning, and (iv) single-agent local Q-learning with improved concept formation rules. In an experiment with a state space larger than 232, it was found that a single-agent local Q-learning agent with improved concept formation rules performed substantially better than a similar agent with less sophisticated concept formation rules and slightly better than a deep Q-learning agent.

Artificial animals

Autonomous agents

Local Q-learning

Efficient concept formation

Adaptive architectures


Fredrik Mäkeläinen

Chalmers, Computer Science and Engineering (Chalmers)

Hampus Torén

Chalmers, Computer Science and Engineering (Chalmers)

Claes Strannegård

Chalmers, Computer Science and Engineering (Chalmers), Data Science

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

03029743 (ISSN) 16113349 (eISSN)

Vol. 10999 140-150
978-3-319-97675-4 (ISBN)

11th International Conference on Artificial General Intelligence, AGI 2018
Prague, Czech Republic,

Subject Categories


Other Mathematics

Computer Science



More information

Latest update