Chapter 47. Learning Reactive Strategies

Key Topics

• Decomposition by Control

• Adaptive Gathering Behaviors

• Modeling Movement

• Learning Shooting Styles

• Other Capabilities

• Evaluation

This chapter uses adaptation to find the most suitable deathmatch strategies. A variety of reinforcement learning (RL) algorithms are used as tools to craft behaviors automatically. This still requires involvement from the AI engineer, but the role involves designing software instead of implementing it. From a designer's point of view, this new system is easier to adjust implicitly but somewhat harder to control explicitly. From the point of view of gameplay, the RL approach allows adaptation, which provides interesting challenges for the players.

The next few pages cover the creation of an AI architecture to learn high-level behaviors, according to evaluative feedback (that is, the moods).

This chapter covers the following topics:

• The practical and theoretical reasons for splitting up the problem into components

• The gathering behaviors optimized using immediate reward only, and collection of statistics to estimate the return

• How the type of movement is learned using Q-learning, which deals with delayed rewards by improving estimates based on others

• An episodic learning algorithm that estimates the benefit of shooting styles at the end of each fight

• Reasons for not using reinforcement learning on other simpler capabilities

• Testing of the system in a game world and an evaluation of the results from a technical perspective

This chapter creates a fully working reactive animat, using reinforcement signals to adapt its behavior.