Dynamic game difficulty balancing
Encyclopedia
Dynamic game difficulty balancing, also known as dynamic difficulty adjustment (DDA) or dynamic game balancing (DGB), is the process of automatically changing parameters, scenarios and behaviors in a video game in real-time, based on the player's ability, in order to avoid them becoming bored (if the game is too easy) or frustrated (if it is too hard). The goal of dynamic difficulty balancing is to keep the user interested from the beginning to the end and to provide a good level of challenge for the user.
Traditionally, game difficulty increases steadily along the course of the game (either in a smooth linear fashion, or through steps represented by the levels). The parameters of this increase (rate, frequency, starting levels) can only be modulated at the beginning of the experience by selecting a difficulty level
. Still, this can lead to a frustrating experience for both experienced and inexperienced gamers, as they attempt to follow a preselected learning or difficulty curve. Dynamic difficulty balancing attempts to remedy this issue by creating a tailor-made experience for each gamer. As the users' skills improve through time (as they make progress via learning), the level of the challenges should also continually increase. However, implementing such elements poses many challenges to game developers; as a result, this method of gameplay is not widespread.
... or any metric used to calculate a game score.
Hunicke
and Chapman’s approach controls the game environment settings in order to make challenges easier or harder. For example, if the game is too hard, the player gets more weapons, recover life points faster or face fewer opponents. Although this approach may be effective, its application can result in implausible situations. A straightforward approach is to combine such "parameters manipulation" to some mechanisms to modify the behavior of the non-player character
s (NPCs) (characters controlled by the computer and usually modeled as intelligent
agents). This adjustment, however, should be made with moderation, to avoid the 'rubber band' effect. One example of this effect in a racing game would involve the AI driver's vehicles becoming significantly faster when behind the player's vehicle, and significantly slower while in front, as if the two vehicles were connected by a large rubber band
.
A traditional implementation of such an agent’s intelligence is to use behavior rules, defined during game development. A typical rule in a fighting game
would state "punch opponent if he is reachable, chase him, otherwise". Extending such approach to include opponent modeling can be made through Spronck et al.′s dynamic scripting
, which assigns to each rule a probability
of being picked. Rule weights can be dynamically updated throughout the game, accordingly to the opponent skills, leading to adaptation to the specific user. With a simple mechanism, rules can be picked that generate tactics that are neither too strong nor too weak for the current player.
Andrade et al. divides the DGB problem into two dimensions: competence (learn as well as possible) and performance (act just as well as necessary). This dichotomy between competence and performance is well known and studied in linguistics
, as proposed by Noam Chomsky
. Their approach faces both dimensions with reinforcement learning
(RL). Offline training is used to bootstrap the learning process. This can be done by letting the agent play against itself (selflearning), other pre-programmed agents, or human players. Then, online learning is used to adapt continually this initially built-in intelligence to each specific human opponent, in order to discover the most suitable strategy to play against him or her. Concerning performance their idea is to find an adequate policy for choosing actions that provide a good game balance, i.e., actions that keep both agent and human player at approximately the same performance level. According to the difficulty the player is facing, the agent chooses actions with high or low expected performance. For a given situation, if the game level is too hard, the agent does not choose the optimal action (provided by the RL framework), but chooses progressively less and less suboptimal actions until its performance is as good as the player’s. Similarly, if the game level becomes too easy, it will choose actions whose values are higher, possibly until it reaches the optimal performance.
Demasi and Cruz built intelligent agents employing genetic algorithm
s techniques to keep alive agents that best fit the user level. Online coevolution is used in order to speed up the learning process. Online coevolution uses pre-defined models (agents with good genetic features) as parents in the genetic operations, so that the evolution is biased by them. These models are constructed by offline training or by hand, when the agent genetic encoding is simple enough.
Other work in the field of DGB is based on the hypothesis that the player-opponent interaction—rather than the audiovisual features, the context or the genre of the game—is the property that contributes the majority of the quality features of entertainment in a computer game. Based on this fundamental assumption, a metric for measuring the real time entertainment value of predator/prey games was introduced, and established as efficient and reliable by validation against human judgment.
Further studies by Yannakakis and Hallam have shown that artificial neural network
s (ANN) and fuzzy neural networks can extract a better estimator of player satisfaction than a human-designed one, given appropriate estimators of the challenge and curiosity (intrinsic qualitative factors for engaging gameplay according to Malone) of the game and data on human players' preferences. The approach of constructing user models of the player of a game that can predict the answers to which variants of the game are more or less fun is defined as Entertainment Modeling. The model is usually constructed using machine learning
techniques applied to game parameters derived from player-game interaction and/or statistical features of player's physiological signals recorded during play. This basic approach is applicable to a variety of games, both computer and physical.
(also called flow) to video games with its 2006 Flash version. The video game design was based on the master's thesis of one of its authors, and was later adapted to PlayStation 3.
The 2008 video game Left 4 Dead
uses a new artificial intelligence technology dubbed "The AI Director". The AI Director is used to procedurally generate a different experience for the players each time the game is played. It monitors individual players performance and how well they work together as a group to pace the game, determining the number of zombies that attack the player and the location of boss infected encounters based on information gathered. Besides pacing the Director also controls some video and audio elements of the game to set a mood for a boss encounter or to draw the players' attention to a certain area.
Valve calls the way the Director is working "Procedural narrative" because instead of having a difficulty level which just ramps up to a constant level, the A.I. analyzes how the players fared in the game so far, and try to add subsequent events that would give them a sense of narrative.
In 2009, the game Resident Evil 5
employed a system called the "Difficulty Scale", unknown to most players as the only mention of it was in the Official Strategy Guide. This system grades the players performance on a number scale from 1 to 10, and adjusts both enemy behavior/attacks used and enemy damage/resistance based on the players' performance (such as deaths, critical attacks, etc.). The selected difficulty levels lock players at a certain number, for example, on Normal difficulty one starts at Grade 4, can move down to Grade 2 if doing poorly, or up to Grade 7 if doing well. The grades between difficulties can overlap.
In the match-3 game Fishdom, the time limit is adjusted based on how well the player performs. The time limit is increased should the player fail a level, making it possible for any player to beat a level after a few tries.
In the 1999 video game Homeworld
, the number of ships that the AI begins with in each mission will be set depending on how powerful the game deems the player's fleet to be. Successful players have larger fleets because they take fewer losses. In this way, a player who is successful over a number of missions will begin to be challenged more and more as the game progresses.
In the video games Fallout: New Vegas
and Fallout 3
, as the player increases in level, tougher variants of enemies, enemies with higher statistics and better weapons, or new enemies will replace older ones to retain a constant difficulty, which can be raised, using a slider, with experience bonuses and vice versa in Fallout 3. This can also be done in New Vegas, but there is no bonus to increasing or decreasing the difficulty.
The Mario Kart
series of games feature items during races that help the individual driver get ahead of opponents. These items are distributed based on a driver's position in a way that is an example of Dynamic Game Difficulty Balancing. For example, a driver near the bottom of the field is likely to get an item that will increase their speed, whereas a driver in first or second place can expect to get these kinds of items rarely.
Traditionally, game difficulty increases steadily along the course of the game (either in a smooth linear fashion, or through steps represented by the levels). The parameters of this increase (rate, frequency, starting levels) can only be modulated at the beginning of the experience by selecting a difficulty level
Difficulty level
In general usage, difficulty level refers to the relative difficulty of completing a task or objective.In computer and video games, the term specifically delineates the ease or difficulty with which an average user may complete a game or a part of a game. Arcade games as well as many early console...
. Still, this can lead to a frustrating experience for both experienced and inexperienced gamers, as they attempt to follow a preselected learning or difficulty curve. Dynamic difficulty balancing attempts to remedy this issue by creating a tailor-made experience for each gamer. As the users' skills improve through time (as they make progress via learning), the level of the challenges should also continually increase. However, implementing such elements poses many challenges to game developers; as a result, this method of gameplay is not widespread.
Dynamic game elements
Some elements of a game that might be changed via dynamic difficulty balancing include:- Speed of enemies
- Health of enemies
- Frequency of enemies
- Frequency of powerups
- Power of player
- Power of enemies
- Duration of gameplay experience
Approaches
Different approaches are found in the literature to address dynamic game difficulty balancing. In all cases, it is necessary to measure, implicitly or explicitly, the difficulty the user is facing at a given moment. This measure can be performed by a heuristic function, which some authors call "challenge function". This function maps a given game state into a value that specifies how easy or difficult the game feels to the user at a specific moment. Examples of heuristics used are:- The rate of successful shots or hits
- The numbers of won and lost pieces
- Life points
- Evolution
- Time to complete some task
... or any metric used to calculate a game score.
Hunicke
Robin Hunicke
Robin Hunicke is a video game designer and producer, currently working at thatgamecompany. Hunicke began her career at Electronic Arts, where she worked on multiple games, including Sims 2 and MySims as a designer, and Boom Blox and Boom Blox: Bash Party as a producer...
and Chapman’s approach controls the game environment settings in order to make challenges easier or harder. For example, if the game is too hard, the player gets more weapons, recover life points faster or face fewer opponents. Although this approach may be effective, its application can result in implausible situations. A straightforward approach is to combine such "parameters manipulation" to some mechanisms to modify the behavior of the non-player character
Non-player character
A non-player character , sometimes known as a non-person character or non-playable character, in a game is any fictional character not controlled by a player. In electronic games, this usually means a character controlled by the computer through artificial intelligence...
s (NPCs) (characters controlled by the computer and usually modeled as intelligent
Game artificial intelligence
Game artificial intelligence refers to techniques used in computer and video games to produce the illusion of intelligence in the behavior of non-player characters . The techniques used typically draw upon existing methods from the field of artificial intelligence...
agents). This adjustment, however, should be made with moderation, to avoid the 'rubber band' effect. One example of this effect in a racing game would involve the AI driver's vehicles becoming significantly faster when behind the player's vehicle, and significantly slower while in front, as if the two vehicles were connected by a large rubber band
Rubber band
A rubber band is a short length of rubber and latex formed in the shape of a loop and is commonly used to hold multiple objects together...
.
A traditional implementation of such an agent’s intelligence is to use behavior rules, defined during game development. A typical rule in a fighting game
Fighting game
Fighting game is a video game genre where the player controls an on-screen character and engages in close combat with an opponent. These characters tend to be of equal power and fight matches consisting of several rounds, which take place in an arena. Players must master techniques such as...
would state "punch opponent if he is reachable, chase him, otherwise". Extending such approach to include opponent modeling can be made through Spronck et al.′s dynamic scripting
Scripting language
A scripting language, script language, or extension language is a programming language that allows control of one or more applications. "Scripts" are distinct from the core code of the application, as they are usually written in a different language and are often created or at least modified by the...
, which assigns to each rule a probability
Probability
Probability is ordinarily used to describe an attitude of mind towards some proposition of whose truth we arenot certain. The proposition of interest is usually of the form "Will a specific event occur?" The attitude of mind is of the form "How certain are we that the event will occur?" The...
of being picked. Rule weights can be dynamically updated throughout the game, accordingly to the opponent skills, leading to adaptation to the specific user. With a simple mechanism, rules can be picked that generate tactics that are neither too strong nor too weak for the current player.
Andrade et al. divides the DGB problem into two dimensions: competence (learn as well as possible) and performance (act just as well as necessary). This dichotomy between competence and performance is well known and studied in linguistics
Linguistics
Linguistics is the scientific study of human language. Linguistics can be broadly broken into three categories or subfields of study: language form, language meaning, and language in context....
, as proposed by Noam Chomsky
Noam Chomsky
Avram Noam Chomsky is an American linguist, philosopher, cognitive scientist, and activist. He is an Institute Professor and Professor in the Department of Linguistics & Philosophy at MIT, where he has worked for over 50 years. Chomsky has been described as the "father of modern linguistics" and...
. Their approach faces both dimensions with reinforcement learning
Reinforcement learning
Inspired by behaviorist psychology, reinforcement learning is an area of machine learning in computer science, concerned with how an agent ought to take actions in an environment so as to maximize some notion of cumulative reward...
(RL). Offline training is used to bootstrap the learning process. This can be done by letting the agent play against itself (selflearning), other pre-programmed agents, or human players. Then, online learning is used to adapt continually this initially built-in intelligence to each specific human opponent, in order to discover the most suitable strategy to play against him or her. Concerning performance their idea is to find an adequate policy for choosing actions that provide a good game balance, i.e., actions that keep both agent and human player at approximately the same performance level. According to the difficulty the player is facing, the agent chooses actions with high or low expected performance. For a given situation, if the game level is too hard, the agent does not choose the optimal action (provided by the RL framework), but chooses progressively less and less suboptimal actions until its performance is as good as the player’s. Similarly, if the game level becomes too easy, it will choose actions whose values are higher, possibly until it reaches the optimal performance.
Demasi and Cruz built intelligent agents employing genetic algorithm
Genetic algorithm
A genetic algorithm is a search heuristic that mimics the process of natural evolution. This heuristic is routinely used to generate useful solutions to optimization and search problems...
s techniques to keep alive agents that best fit the user level. Online coevolution is used in order to speed up the learning process. Online coevolution uses pre-defined models (agents with good genetic features) as parents in the genetic operations, so that the evolution is biased by them. These models are constructed by offline training or by hand, when the agent genetic encoding is simple enough.
Other work in the field of DGB is based on the hypothesis that the player-opponent interaction—rather than the audiovisual features, the context or the genre of the game—is the property that contributes the majority of the quality features of entertainment in a computer game. Based on this fundamental assumption, a metric for measuring the real time entertainment value of predator/prey games was introduced, and established as efficient and reliable by validation against human judgment.
Further studies by Yannakakis and Hallam have shown that artificial neural network
Artificial neural network
An artificial neural network , usually called neural network , is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes...
s (ANN) and fuzzy neural networks can extract a better estimator of player satisfaction than a human-designed one, given appropriate estimators of the challenge and curiosity (intrinsic qualitative factors for engaging gameplay according to Malone) of the game and data on human players' preferences. The approach of constructing user models of the player of a game that can predict the answers to which variants of the game are more or less fun is defined as Entertainment Modeling. The model is usually constructed using machine learning
Machine learning
Machine learning, a branch of artificial intelligence, is a scientific discipline concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data, such as from sensor data or databases...
techniques applied to game parameters derived from player-game interaction and/or statistical features of player's physiological signals recorded during play. This basic approach is applicable to a variety of games, both computer and physical.
Uses in recent video games
The video game Flow was notable for popularizing the application of mental immersionFlow (psychology)
Flow is the mental state of operation in which a person in an activity is fully immersed in a feeling of energized focus, full involvement, and success in the process of the activity. Proposed by Mihály Csíkszentmihályi, the positive psychology concept has been widely referenced across a variety of...
(also called flow) to video games with its 2006 Flash version. The video game design was based on the master's thesis of one of its authors, and was later adapted to PlayStation 3.
The 2008 video game Left 4 Dead
Left 4 Dead
Left 4 Dead is a cooperative first-person shooter video game. It was developed by Turtle Rock Studios, which was purchased by Valve Corporation during development. The game uses Valve's proprietary Source engine, and is available for Microsoft Windows, Xbox 360 and Mac OS X...
uses a new artificial intelligence technology dubbed "The AI Director". The AI Director is used to procedurally generate a different experience for the players each time the game is played. It monitors individual players performance and how well they work together as a group to pace the game, determining the number of zombies that attack the player and the location of boss infected encounters based on information gathered. Besides pacing the Director also controls some video and audio elements of the game to set a mood for a boss encounter or to draw the players' attention to a certain area.
Valve calls the way the Director is working "Procedural narrative" because instead of having a difficulty level which just ramps up to a constant level, the A.I. analyzes how the players fared in the game so far, and try to add subsequent events that would give them a sense of narrative.
In 2009, the game Resident Evil 5
Resident Evil 5
Resident Evil 5, known in Japan as , is a survival horror third-person shooter video game developed and published by Capcom. The game is the seventh installment in the Resident Evil survival horror series, and was released on March 5, 2009 in Japan and on March 13, 2009 in North America and Europe...
employed a system called the "Difficulty Scale", unknown to most players as the only mention of it was in the Official Strategy Guide. This system grades the players performance on a number scale from 1 to 10, and adjusts both enemy behavior/attacks used and enemy damage/resistance based on the players' performance (such as deaths, critical attacks, etc.). The selected difficulty levels lock players at a certain number, for example, on Normal difficulty one starts at Grade 4, can move down to Grade 2 if doing poorly, or up to Grade 7 if doing well. The grades between difficulties can overlap.
In the match-3 game Fishdom, the time limit is adjusted based on how well the player performs. The time limit is increased should the player fail a level, making it possible for any player to beat a level after a few tries.
In the 1999 video game Homeworld
Homeworld
Homeworld is a real-time strategy computer game released on September 28, 1999, developed by Relic Entertainment and published by Sierra Entertainment. It was the first fully three-dimensional RTS. In 2003, Relic released the source code for Homeworld...
, the number of ships that the AI begins with in each mission will be set depending on how powerful the game deems the player's fleet to be. Successful players have larger fleets because they take fewer losses. In this way, a player who is successful over a number of missions will begin to be challenged more and more as the game progresses.
In the video games Fallout: New Vegas
Fallout: New Vegas
Fallout: New Vegas is a first person action role-playing video game in the Fallout series developed by Obsidian Entertainment, and published by Bethesda Softworks. The game is based in a post-apocalyptic environment in and around Las Vegas, Nevada...
and Fallout 3
Fallout 3
Fallout 3 is an action role-playing game released by Bethesda Game Studios, and the third major installment in the Fallout series. The game was released in North America, Europe and Australia in October 2008, and in Japan in December 2008 for Microsoft Windows, PlayStation 3 and Xbox 360...
, as the player increases in level, tougher variants of enemies, enemies with higher statistics and better weapons, or new enemies will replace older ones to retain a constant difficulty, which can be raised, using a slider, with experience bonuses and vice versa in Fallout 3. This can also be done in New Vegas, but there is no bonus to increasing or decreasing the difficulty.
The Mario Kart
Mario Kart
is a series of go-kart-style racing video games developed by Nintendo as a series of spin-offs from its trademark Mario series of platformer adventure-style video games...
series of games feature items during races that help the individual driver get ahead of opponents. These items are distributed based on a driver's position in a way that is an example of Dynamic Game Difficulty Balancing. For example, a driver near the bottom of the field is likely to get an item that will increase their speed, whereas a driver in first or second place can expect to get these kinds of items rarely.
See also
- Difficulty levelDifficulty levelIn general usage, difficulty level refers to the relative difficulty of completing a task or objective.In computer and video games, the term specifically delineates the ease or difficulty with which an average user may complete a game or a part of a game. Arcade games as well as many early console...
- Nonlinear gameplay
- Game balanceGame balanceGame balance is a concept in game design describing fairness or balance of power in a game between multiple players or strategic options. Each team or person would be equally matched in every aspect.- Overview :...
- Game artificial intelligenceGame artificial intelligenceGame artificial intelligence refers to techniques used in computer and video games to produce the illusion of intelligence in the behavior of non-player characters . The techniques used typically draw upon existing methods from the field of artificial intelligence...
- Flow (psychology)Flow (psychology)Flow is the mental state of operation in which a person in an activity is fully immersed in a feeling of energized focus, full involvement, and success in the process of the activity. Proposed by Mihály Csíkszentmihályi, the positive psychology concept has been widely referenced across a variety of...
External links
-
Dynamic Difficulty Adjustment - Game Ontology Wikidead link