FoCAS Sponsor Best Paper Award at ALA 2015

Building on a continuing relationship with the ALA workshop (Adaptive Learning Agents), FoCAS sponsored a Best Paper Award for this workshop held at AAMAS 2015 in Istanbul on 5 May 2015.

The winning paper award was presented to Anna Harutyunan from Vrije Universiteit Brussel for her paper “Off-Policy Reward Shaping with Ensembles” by Anna Harutyunyan, Tim Brys, Peter Vrancx and Anne Nowe.

ALA Best Paper Winner

Emma Hart presenting Anna Harutyunyan with the FoCAS Best Paper Award

Potential-based reward shaping (PBRS) is an effective and popular technique to speed up reinforcement learning by leveraging domain knowledge. While PBRS is proven to always preserve optimal policies, its effect on learning speed is determined by the quality of its potential function, which, in turn, depends on both the underlying heuristic and the scale. Knowing which heuristic will prove effective requires testing the options beforehand, and determining the appro- priate scale requires tuning, both of which introduce additional sample complexity. We formulate a PBRS framework that improves learning speed, but does not incur extra sample complexity. For this, we propose to simultaneously learn an ensemble of policies, shaped w.r.t. many heuristics and on a range of scales. The target policy is then obtained by voting. The ensemble needs to be able to efficiently and reliably learn off-policy: requirements fulfilled by the recent Horde architecture, which we take as our basis. We demonstrate empirically that (1) our ensemble policy outperforms both the base policy, and its single-heuristic components, and (2) an ensemble over a general range of scales performs at least as well as one with optimally tuned components.

This workshop is a continuation of the long running AAMAS series of workshops on adaptive agents, now in its fourteenth year. ALA aims to bring together scientists from different areas of computer science, software engineering, biology as well as cognitive and social sciences, as well as those from other fields studying similar concepts (e.g., game theory, bio-inspired control, mechanism design). The ALA workshop focusses on agents and multi-agent systems which employ learning or adaptation.


FoCAS are sponsoring a Best Paper Award at the ALA (Adaptive Learning Agents) 2015 Workshop held in conjunction with AAMAS 2015.

Details of the CFP are below:

Adaptive Learning Agents (ALA) encompasses diverse fields such as Computer Science, Software Engineering, Biology, as well as Cognitive and Social Sciences. The ALA workshop will focus on agents and multiagent systems which employ learning or adaptation.

This workshop is a continuation of the long running AAMAS series of workshops on adaptive agents, now in its fourteenth year.

The goal of this workshop is to increase awareness and interest in adaptive agent research, encourage collaboration and give a representative overview of current research in the area of adaptive and learning agents and multiagent systems. It aims at bringing together not only scientists from different areas of computer science (e.g., agent architectures, reinforcement learning, and evolutionary algorithms) but also from different fields studying similar concepts (e.g., game theory, bio-inspired control, mechanism design).

The workshop will serve as an inclusive forum for the discussion on ongoing or completed work in both theoretical and practical issues of adaptive and learning agents and multiagent systems.

This workshop will focus on all aspects of adaptive and learning agents and multiagent systems with a particular emphasis on how to modify established learning techniques and/or create new learning paradigms to address the many challenges presented by complex real-world problems. The topics of interest include but are not limited to:

  • Novel combinations of reinforcement and supervised learning approaches
  • Integrated learning approaches that work with other agent reasoning modules like negotiation, trust models, coordination, etc.
  • Supervised multiagent learning
  • Reinforcement learning (single and multiagent)
  • Planning (single and multiagent)
  • Reasoning (single and multiagent)
  • Distributed learning
  • Adaptation and learning in dynamic environments
  • Evolution of agents in complex environments
  • Co-evolution of agents in a multiagent setting
  • Cooperative exploration and learning to cooperate and collaborate
  • Learning trust and reputation
  • Communication restrictions and their impact on multiagent coordination
  • Design of reward structure and fitness measures for coordination
  • Scaling learning techniques to large systems of learning and adaptive agents
  • Emergent behaviour in adaptive multiagent systems
  • Game theoretical analysis of adaptive multiagent systems
  • Neuro-control in multiagent systems
  • Bio-inspired multiagent systems
  • Applications of adaptive and learning agents and multiagents systems to real world complex systems
  • Learning of Co-ordination

Accepted papers from the workshop will be eligible to be extended for inclusion in a special issue journal.

Important Dates

Submission Deadline: February 11, 2015
Notification of acceptance: March 6, 2015
Camera-ready copies: March 19, 2015
Early registration: t.b.a.
Workshop: May 4-5, 2015

Submission Details

Papers should be submitted by February 11, 2015, through the EasyChair submission system.
Submissions may be up to 8 pages in the ACM proceedings format (i.e., the same as AAMAS papers in the main conference track). Accepted work will be allocated time for oral presentation during the two day workshop. Papers accepted at the workshop will also be eligible for inclusion in a planned special issue journal published after the workshop.

Invited Talk

To be announced


This year’s workshop is organized by:
Daan Bloembergen (University of Liverpool, UK)
Daniel Hennes (European Space Agency, Katwijk, NL)
Logan Yliniemi (Oregon State University, Corvallis, USA)

Senior Steering Committee Members:
Daniel Kudenko (University of York, UK)
Ann Nowé (Vrije Universiteit Brussels, Belgium)
Sandip Sen (University of Tulsa, USA)
Peter Stone (University of Texas at Austin, USA)
Matthew Taylor (Washington State University, USA)
Kagan Tumer (Oregon State University, USA)
Karl Tuyls (University of Liverpool, UK)


If you have any questions about the ALA workshop, please contact the organizers at:
ala.workshop.2015 AT