Cooperative Multi-Agent Learning:
The State of the Art
Liviu Panait and Sean Luke
George Mason University
Abstract
Cooperative multi-agent systems are ones in which several agents attempt, through their interaction, to jointly
solve tasks or to maximize utility. Due to the interactions among the agents, multi-agent problem complexity can
rise rapidly with the number of agents or their behavioral sophistication. The challenge this presents to the task
of programming solutions to multi-agent systems problems has spawned increasing interest in machine learning
techniques to automate the search and optimization process.
We provide a broad survey of the cooperative multi-agent learning literature. Previous surveys of this area have
largely focused on issues common to specific subareas (for example, reinforcement learning or robotics). In this
survey we attempt to draw from multi-agent learning work in a spectrum of areas, including reinforcement learning,
evolutionary computation, game theory, complex systems, agent modeling, and robotics.
We find that this broad view leads to a division of the work into two categories, each with its own special is-
sues: applying a single learner to discover joint solutions to multi-agent problems (team learning), or using multiple
simultaneous learners, often one per agent (concurrent learning). Additionally, we discuss direct and indirect commu-
nication in connection with learning, plus open issues in task decomposition, scalability, and adaptive dynamics. We
conclude with a presentation of multi-agent learning problem domains, and a list of multi-agent learning resources.
1 Introduction
In recent years there has been increased interest in decentralized approaches to solving complex real-world problems.
Many such approaches fall into the area of distributed systems, where a number of entities work together to coopera-
tively solve problems. The combination of distributed systems and artificial intelligence (AI) is collectively known as
distributed artificial intelligence (DAI). Tr