12.3 Models of bounded rationality and learning in games
7 min read•july 30, 2024
challenges the idea of perfect decision-making in games. It recognizes that players have limits on time, info, and brainpower. This leads to different outcomes than what classical game theory predicts, as people use shortcuts and learn as they go.
Models of learning in games show how players adjust their strategies over time. Some focus on reinforcement from past payoffs, while others look at beliefs about opponents. Hybrid models combine both approaches. These ideas help explain real-world behavior in strategic situations.
Bounded Rationality in Games
Concept and Implications
Top images from around the web for Concept and Implications
Herbert Simon: Racionalidad Limitada - Percepciones Económicas View original
Is this image relevant?
Learning Approaches | Introduction to Psychology – Reinke View original
Is this image relevant?
Making Decisions in Different Organizations | Organizational Behavior / Human Relations View original
Is this image relevant?
Herbert Simon: Racionalidad Limitada - Percepciones Económicas View original
Is this image relevant?
Learning Approaches | Introduction to Psychology – Reinke View original
Is this image relevant?
1 of 3
Top images from around the web for Concept and Implications
Herbert Simon: Racionalidad Limitada - Percepciones Económicas View original
Is this image relevant?
Learning Approaches | Introduction to Psychology – Reinke View original
Is this image relevant?
Making Decisions in Different Organizations | Organizational Behavior / Human Relations View original
Is this image relevant?
Herbert Simon: Racionalidad Limitada - Percepciones Económicas View original
Is this image relevant?
Learning Approaches | Introduction to Psychology – Reinke View original
Is this image relevant?
1 of 3
- Bounded rationality is the idea that decision-makers have limited cognitive abilities and face constraints such as time, information, and computational capacity when making decisions
- Contrasts with the assumption of perfect rationality in classical game theory
- Bounded rationality can lead to deviations from the predictions of classical game theory
- Players may not always choose the optimal strategy or reach the
- Instead, they may use heuristics, , or other simplified decision rules
- The implications of bounded rationality for game-theoretic modeling include:
- The need to consider the cognitive limitations of players and their impact on strategic behavior
- The potential for multiple equilibria or non-equilibrium outcomes in games
- The importance of learning and adaptation in shaping the dynamics of strategic interactions
- The role of heuristics and simplified decision rules in guiding player behavior
Models and Goals
- Models of bounded rationality aim to capture these limitations and provide more realistic descriptions of human decision-making in strategic situations
- Often incorporate cognitive constraints, learning, and adaptation
- Examples include , , , and
- The goals of bounded rationality models include:
- Explaining deviations from the predictions of classical game theory and Nash equilibrium
- Providing more accurate predictions of human behavior in strategic situations
- Incorporating the role of cognitive limitations, learning, and adaptation in shaping strategic decision-making
- Offering insights into the design of institutions, markets, and incentives that account for bounded rationality
Models of Learning in Games
Reinforcement and Belief Learning
- is a model where players adjust their strategies based on the payoffs they receive
- Players are more likely to repeat strategies that have yielded high payoffs in the past and less likely to use strategies that have led to low payoffs
- Example: A firm that experiences increased profits after a price cut is more likely to continue using low-price strategies in the future
- is a model where players form beliefs about the strategies of their opponents based on observed behavior and update these beliefs over time
- Players choose their strategies based on their current beliefs about the likelihood of different opponent actions
- Example: In a repeated prisoner's dilemma, a player who observes their opponent cooperating in previous rounds may form the belief that cooperation is more likely and adjust their strategy accordingly
Hybrid and Adaptive Learning Models
- (EWA) learning is a hybrid model that combines elements of reinforcement and belief learning
- Players update both their propensities to play different strategies (reinforcement) and their beliefs about opponent strategies (belief) based on past experience
- EWA can capture both the direct effect of payoffs on strategy choice and the indirect effect of beliefs about opponent behavior
- , such as , assume that players best-respond to a weighted average of their opponents' past actions
- More recent actions receive greater weight in the player's decision-making process
- Adaptive learning models can capture the idea that players place more emphasis on recent experiences when forming expectations about opponent behavior
- Other learning models in games include:
- , where players copy the strategies of successful opponents
- , where players adjust their strategies based on whether their payoffs exceed or fall short of an aspiration level
- Example: A firm that observes a competitor's successful marketing campaign may imitate this strategy in an attempt to improve its own performance
Bounded Rationality vs Nash Equilibrium
Explaining Deviations
- Models of bounded rationality can help explain why observed behavior in games often deviates from the predictions of Nash equilibrium, which assumes perfect rationality
- Cognitive hierarchy models assume that players have different levels of strategic sophistication, with some players being more sophisticated than others
- These models can explain deviations from Nash equilibrium by accounting for the presence of less sophisticated players who may not best-respond to their opponents' strategies
- Example: In a p-beauty contest game, where players choose numbers between 0 and 100 and the winner is the one closest to 2/3 of the average, the Nash equilibrium prediction is that all players choose 0. However, experiments show that players often choose higher numbers, which can be explained by the presence of less sophisticated players who do not fully iterate the best-response reasoning process
- Quantal response equilibrium (QRE) is a model that allows for stochastic choice, where players choose strategies with probabilities that are increasing in their expected payoffs
- QRE can explain deviations from Nash equilibrium by allowing for "noisy" decision-making and the possibility of suboptimal choices
- Example: In a game with multiple Nash equilibria, QRE can explain why players might not always coordinate on the most efficient equilibrium, as the probability of choosing each strategy depends on its relative payoff
Accounting for Heterogeneity in Strategic Thinking
- Level-k thinking models assume that players have different levels of strategic reasoning, with level-0 players choosing randomly, level-1 players best-responding to level-0, and so on
- These models can explain deviations from Nash equilibrium by capturing the heterogeneity in players' strategic thinking
- Example: In a game of rock-paper-scissors, the Nash equilibrium prediction is that players will choose each action with equal probability. However, level-k models can explain why some players might choose actions that exploit the anticipated choices of less sophisticated opponents
- Cursed equilibrium is a model where players fail to fully account for the correlation between their opponents' actions and their private information
- This can lead to deviations from Nash equilibrium predictions, particularly in games with incomplete information
- Example: In a common-value auction, where the value of the auctioned item is the same for all bidders but unknown at the time of bidding, cursed equilibrium can explain why players might overbid and fall prey to the winner's curse, as they fail to fully account for the information conveyed by winning the auction
Predictive Power of Bounded Rationality Models
Evaluating Predictive Power and Empirical Validity
- The predictive power of a model refers to its ability to accurately forecast behavior in new or out-of-sample situations. Empirical validity concerns the extent to which a model's predictions match observed data from experiments or real-world settings
- To evaluate the predictive power and empirical validity of models of bounded rationality and learning, researchers compare the models' predictions with data from controlled experiments or field studies
- This involves designing experiments that can distinguish between the predictions of different models and collecting data on actual player behavior
- Example: Researchers might design an experiment to test the predictions of reinforcement learning and belief learning models in a repeated game, and compare the models' performance in predicting the observed behavior of participants
Model Comparison and Cross-Validation
- Model comparison techniques, such as the Akaike information criterion (AIC) or the Bayesian information criterion (BIC), can be used to assess the relative fit of different models to the data while accounting for model complexity
- Models with lower AIC or BIC values are preferred, as they strike a balance between goodness-of-fit and parsimony
- Example: When comparing the performance of different learning models in a game, researchers might calculate the AIC or BIC values for each model and select the one with the lowest value as the best-fitting model
- Cross-validation methods, such as k-fold cross-validation or leave-one-out cross-validation, can be used to assess the out-of-sample predictive performance of models
- These methods involve splitting the data into training and testing sets, fitting the models on the training set, and evaluating their predictions on the testing set
- Example: In a study comparing the predictive power of different bounded rationality models, researchers might use k-fold cross-validation to estimate the models' performance on unseen data and select the model with the highest average performance across the folds
Robustness and Generalizability
- Robustness checks, such as testing the models' predictions under different experimental conditions or with different subject pools, can help establish the generalizability and external validity of the models
- Example: To test the robustness of a bounded rationality model, researchers might replicate the experiment with different payoff structures, different subject populations (e.g., students vs. professionals), or in different cultural contexts
- The empirical validity of models of bounded rationality and learning is an ongoing area of research, with different models performing better in different contexts
- Comparing the performance of multiple models across a range of strategic situations is important for understanding their strengths and limitations
- Example: A model that performs well in predicting behavior in simple games may not necessarily generalize to more complex strategic environments, highlighting the need for testing models across a variety of contexts
Key Terms to Review (22)
Adaptive Learning Models: Adaptive learning models are frameworks that adjust the strategies and decisions of players in a game based on their previous experiences and outcomes. These models acknowledge that individuals often operate under bounded rationality, meaning they make decisions with limited information and cognitive resources. As players interact over time, they learn from their successes and failures, refining their strategies to better respond to the actions of others in the game environment.
Aspiration-based learning: Aspiration-based learning is a concept in game theory that describes how players adjust their strategies based on aspirations or goals they set for themselves. Instead of trying to optimize their payoffs based on complete information, players use their aspirations to make decisions, focusing on achieving specific outcomes that they consider satisfactory or desirable. This approach highlights the limitations of rational decision-making by illustrating how players may adopt adaptive strategies that help them learn from past experiences while striving to meet their aspirations.
Behavioral Economics: Behavioral economics is a field that combines insights from psychology and economics to understand how individuals make decisions that deviate from traditional economic theory. It highlights the influence of cognitive limitations and emotional factors on decision-making processes, emphasizing that people often act irrationally in predictable ways. This field also explores models of bounded rationality, which reflect how individuals learn and adapt their strategies in various scenarios, especially in games.
Belief Learning: Belief learning is a process through which players in a game update their beliefs about other players' strategies based on past experiences and observations. This adaptive learning mechanism helps individuals make more informed decisions in uncertain environments, leading to more effective strategies over time. In contexts where players have limited rationality, belief learning allows them to adjust their expectations and choices as they gather new information from the outcomes of previous interactions.
Bounded rationality: Bounded rationality refers to the idea that individuals, when making decisions, are limited by their cognitive abilities, available information, and time constraints. This concept highlights that humans often rely on simplifying strategies or heuristics rather than fully rational approaches, leading to decisions that may not always align with traditional economic models of rational choice.
Cognitive Hierarchy Models: Cognitive hierarchy models are frameworks used to understand how individuals make decisions in strategic situations by assuming different levels of reasoning among players. These models account for bounded rationality, where not all players think alike or possess the same level of strategic sophistication. By incorporating varying levels of thought and beliefs about others’ reasoning, cognitive hierarchy models help predict behavior in games more accurately than traditional equilibrium concepts.
Cursed equilibrium: Cursed equilibrium is a concept in game theory that refers to a situation where players fail to reach a mutually beneficial outcome due to incorrect beliefs or misconceptions about others' preferences or strategies. This occurs because players assume that their opponents are more rational or benevolent than they actually are, leading them to make suboptimal decisions. As a result, cursed equilibrium highlights the limitations of bounded rationality and learning processes within strategic interactions.
Daniel Kahneman: Daniel Kahneman is a renowned psychologist known for his groundbreaking work in behavioral economics and decision-making, particularly regarding how people perceive risk and make choices under uncertainty. His research has profoundly influenced the understanding of human behavior, revealing that individuals often rely on cognitive shortcuts, leading to systematic biases in judgment and decision-making.
Experience-weighted attraction: Experience-weighted attraction is a concept in game theory that describes how individuals adjust their preferences based on their past experiences in decision-making scenarios. This approach helps to model how players learn over time, as they weigh their experiences more heavily when determining which strategies to adopt, leading to more refined decision-making. By considering both successful and unsuccessful outcomes, players can update their strategies and adapt to changing environments in games.
Fictitious play: Fictitious play is a learning process in game theory where players make decisions based on the historical behavior of their opponents, assuming that these behaviors will continue in the future. This approach models bounded rationality by allowing players to learn and adapt over time, rather than requiring them to possess complete knowledge of the game or their opponents' strategies. Fictitious play provides a framework for understanding how players can converge to Nash equilibrium through iterative best-response strategies.
Herbert Simon: Herbert Simon was a pioneering American psychologist and economist who made significant contributions to the understanding of decision-making and problem-solving, particularly through the lens of bounded rationality. His work emphasized that individuals make decisions based on limited information and cognitive constraints, which plays a critical role in artificial intelligence and multi-agent systems as well as in models of learning in strategic environments.
Imitation learning: Imitation learning is a type of learning where an agent learns to perform tasks by observing and mimicking the behavior of others. This concept is particularly relevant in scenarios where individuals or agents do not have complete information or the capability to explore all possible strategies, which connects to bounded rationality. Imitation learning allows agents to adapt and improve their decision-making processes based on the observed successes and failures of others, highlighting the importance of social learning in strategic environments.
Level-k thinking models: Level-k thinking models are a framework in game theory that describe how individuals strategize based on their beliefs about others' thought processes. In these models, players are categorized by levels of reasoning, where a 'level-0' player chooses actions randomly, a 'level-1' player best responds to level-0 players, a 'level-2' player best responds to level-1 players, and so on. This hierarchy illustrates bounded rationality by showing how different players utilize varying degrees of strategic thinking when making decisions in games.
Market behavior: Market behavior refers to the way in which participants in a market make decisions and interact with one another, particularly under conditions of uncertainty and competition. It encompasses the patterns of actions and reactions among buyers and sellers, influenced by various factors such as preferences, strategies, and available information. Understanding market behavior is crucial for analyzing economic outcomes and helps explain why individuals or groups might deviate from traditional rationality in their decision-making processes.
Nash equilibrium: Nash equilibrium is a concept in game theory where no player can benefit from changing their strategy while the other players keep theirs unchanged. This situation arises when each player's strategy is optimal given the strategies of all other players, leading to a stable state in strategic interactions.
Observational learning: Observational learning is a process through which individuals acquire new behaviors or information by watching others. This type of learning emphasizes the importance of modeling and imitation, allowing individuals to learn without direct experience or reinforcement. In the context of bounded rationality and learning in games, observational learning plays a significant role in shaping players' strategies and decision-making processes based on the observed actions of others.
Quantal response equilibrium: Quantal response equilibrium is a solution concept in game theory that generalizes Nash equilibrium by incorporating the idea that players may make decisions based on probabilistic responses to their opponents' strategies. In this framework, players' actions are influenced by their beliefs about others’ strategies and the cognitive limitations they face, leading to stochastic choices rather than deterministic ones. This concept connects to decision-making biases, computational complexity, and models of bounded rationality as it reflects how real-world decision-making often deviates from perfect rationality.
Reinforcement Learning: Reinforcement learning is a type of machine learning where an agent learns to make decisions by taking actions in an environment to maximize cumulative rewards over time. This learning process involves exploration and exploitation, where the agent must balance trying new actions and using known ones that yield high rewards. It's deeply tied to concepts of decision-making biases and cognitive limitations, as well as applications in AI, especially when multiple agents interact with each other in complex environments.
Satisficing: Satisficing is a decision-making strategy that aims for a satisfactory or adequate result, rather than the optimal one. It reflects the idea that individuals often settle for a choice that meets their minimum requirements due to constraints like limited information, time, or cognitive resources. This approach recognizes the challenges in achieving perfect rationality and highlights how people navigate complex decisions by choosing options that are 'good enough' instead of the best possible.
Social preferences: Social preferences refer to the ways in which individuals' utility is influenced not only by their own outcomes but also by the outcomes of others. This concept acknowledges that people often care about fairness, altruism, and cooperation, which can significantly affect decision-making and strategic interactions in various settings. Understanding social preferences is crucial for analyzing behavior in experiments and how individuals learn and adapt their strategies based on social interactions.
Stochastic Stability: Stochastic stability refers to the resilience of a particular equilibrium in a dynamic system when subjected to random perturbations or noise. It focuses on how likely certain strategies or behaviors are to persist over time in environments where players face uncertainty and adapt their choices based on past experiences. This concept is particularly important in understanding how networks evolve and how boundedly rational agents learn and adjust their strategies in interactive settings.
Weighted fictitious play: Weighted fictitious play is a learning process in game theory where players adjust their strategies based on the historical frequencies of opponents' actions, giving more weight to recent observations. This approach is a modification of standard fictitious play, aiming to better reflect how boundedly rational players learn and adapt over time. By incorporating weights, players can improve their decision-making in strategic interactions where past experiences influence future choices.