Efficient concept formation in large state spaces
Paper in proceedings, 2018

General autonomous agents must be able to operate in previously unseen worlds with large state spaces. To operate successfully in such worlds, the agents must maintain their own models of the environment, based on concept sets that are several orders of magnitude smaller. For adaptive agents, those concept sets cannot be fixed, but must adapt continuously to new situations. This, in turn, requires mechanisms for forming and preserving those concepts that are critical to successful decision-making, while removing others. In this paper we compare four general algorithms for learning and decision-making: (i) standard Q-learning, (ii) deep Q-learning, (iii) single-agent local Q-learning, and (iv) single-agent local Q-learning with improved concept formation rules. In an experiment with a state space larger than 232, it was found that a single-agent local Q-learning agent with improved concept formation rules performed substantially better than a similar agent with less sophisticated concept formation rules and slightly better than a deep Q-learning agent.

Artificial animals

Autonomous agents

Local Q-learning

Efficient concept formation

Adaptive architectures

Author

Fredrik Mäkeläinen

Chalmers, Computer Science and Engineering (Chalmers)

Hampus Torén

Chalmers, Computer Science and Engineering (Chalmers)

Claes Strannegård

Chalmers, Computer Science and Engineering (Chalmers), Data Science

Proc. 11th International Conference on Artificial General Intelligence

Vol. 10999 140-150

11th International Conference on Artificial General Intelligence, AGI 2018
Prague, Czech Republic,

Subject Categories

Philosophy

Other Mathematics

Computer Science

DOI

10.1007/978-3-319-97676-1_14

More information

Latest update

1/15/2019