Recursive Lexicographical Search: Finding all Markov Perfect - PowerPoint PPT Presentation

Introduction: Computing All Markov Perfect Equilibria How to find all equilibria? PART I : Solving the Leapfrogging Model with RLS Problems with Value Function Iteration PART II: Recursive Lexicographical Search in General DDGs Existing methods for finding ALL MPE Conclusions Our Approach and Road Map for the Rest of the Talk Relation to Subgame Perfection Kuhn (1953) and Selten (1965) showed that standard backward induction on the game tree (the extensive form representation of the game) can be used to compute all subgame perfect equilibria of finite games In a DDG, we can represent the directionality of the game via a directed acyclic graph (DAG). While every game tree is a DAG, the DAGs that represent directionality in a dynamic game are not game trees. Instead, the DAG summarizes the directionality of the game in terms of the state space instead of the temporal ordering implied by the game tree. State recursion can be viewed as a generalization of standard backward induction, but a type of backward induction that is performed on the DAG rather than on the game tree. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria How to find all equilibria? PART I : Solving the Leapfrogging Model with RLS Problems with Value Function Iteration PART II: Recursive Lexicographical Search in General DDGs Existing methods for finding ALL MPE Conclusions Our Approach and Road Map for the Rest of the Talk Recursive Lexicographical Search (RLS) State recursion can be used to find a single MPE of a dynamic game G It depends on a specification of an equilibrium selection rule for selecting a particular MPE at each state of the game that constitute the behavior strategies used by the players Recursive lexicographical search (RLS) is an algorithm that repeatedly invokes state recursion in an e ffi cient way to compute all MPE of the DDG by systematically cycling through all feasible equilibrium selection rules (ESRs) for each of the component stage games of the DDG. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria How to find all equilibria? PART I : Solving the Leapfrogging Model with RLS Problems with Value Function Iteration PART II: Recursive Lexicographical Search in General DDGs Existing methods for finding ALL MPE Conclusions Our Approach and Road Map for the Rest of the Talk Relation to Literature on Finitely Repeated Games The idea of how multiple equilibria of a stage game can be used to construct a much larger set of equilibria in the overall game was used by Benoit and Krishna (1985) to show that a version of the Folk Theorem can hold in finitely repeated games. The prevailing view prior to their work was that the extreme multiplicity of equilibria implied by the Folk Theorem for infinitely repeated games cannot happen because backward induction from the last period results in a unique equilibrium in a finitely repeated game. Benoit and Krishna showed that multiplicity of equilibria in the stage games can be used to create a much larger set of subgame perfect equilibria in the finitely repeated game, so the Folk Theorem can emerge if the time horizon is su ffi ciently large. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria How to find all equilibria? PART I : Solving the Leapfrogging Model with RLS Problems with Value Function Iteration PART II: Recursive Lexicographical Search in General DDGs Existing methods for finding ALL MPE Conclusions Our Approach and Road Map for the Rest of the Talk Benoit and Krishna vs RLS However Benoit and Krishna did not propose an algorithm to enumerate all possible subgame perfect equilibria of a finitely repeated game, whereas the RLS algorithm we propose can find and enumerate all such equilibria. Though we do not claim that all dynamic games will have exploitable directional structure, we show there is a sense in which the RLS algorithm can approximate the set of all MPE for a wide class of finite and infinite-horizon dynamic games, even if there is no exploitable directionality in the game other than the passage of time. We show how backward induction performed in the right way can approximate all MPE of a fairly broad class of infinite horizon games. We view this as an analog of Benoit and Krishnas Folk Theorem approximation result for finitely repeated games. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria How to find all equilibria? PART I : Solving the Leapfrogging Model with RLS Problems with Value Function Iteration PART II: Recursive Lexicographical Search in General DDGs Existing methods for finding ALL MPE Conclusions Our Approach and Road Map for the Rest of the Talk Road Map for the Talk Part I: Example of a DDG (The Leapfroging Model) 1 Illustrate RLS by using it to find all MPE of a dynamic model of Bertrand price competition with leapfrogging investments. 2 Some theoretical results and numerical simulations 3 Danger of imposing symmetry 4 Limitations of homotopy parameter methods 5 Implications for empirical work Part II: The general case Rigorously define the concept of a general class of directional dynamic games Introduce the state recursion algorithm and show that it can compute a single MPE of the game G Introduce the recursive lexicographical search algorithm and show that it can find all MPE of G Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions PART I A DDG example: The Dynamics of Bertrand Price Competition with Cost-Reducing Investments (Iskhakov, Rust, Schjerning (2013)) Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Example: The leapfrogging model Basic Setup Discrete time, infinite horizon ( t = 1 , 2 , 3 , . . . ) Two firms, homogenous/di ff erentiated goods, no entry or exit Each firm maximizes expected discounted profits Bertrand competition: set product prices (simultaneously) Investment decision : Whether to invest in state of the art production technology Pay investment cost of K ( c ) to obtain marginal cost c � 0 Time to build: state of the art technology is operational after a one period lag State of the art costs follows exogenous Markov process and only improves Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Example: The leapfrogging model - cont. Timing of cost reducing investment decisions 1 Simultaneous moves : Fully directional, d = ( c 1 , c 2 , c ) 2 Alternating moves : The right to move, m , follows a Markov process (deterministic alternation as a special case). m is clearly a non-directional, but game still have a directional component, d = ( c 1 , c 2 , c ) Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions State space of the game: a “quarter pyramid” S = { ( c 1 , c 2 , c ) | c 1 ≥ c , c 2 ≥ c , c ∈ [0 , c ] } Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions End Games States Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions No Investment End Games States Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Multiple Equilibria End Games States Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Second order best response function, η = 0 End Game Equilibria (c 1 ,c 2 )=(0.714286,2.14286) k=7 beta=0.95 1 0.9 0.8 2nd order best response function 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Firm 1’s probability of investing Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Calculated Equilibria for End Games States Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Recursion Level 1 States Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Recursion Level 1 States, isolated Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions No Investment Recursion Level 1 States Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Multiple Equilibria Recursion Level 1 States Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Calculated Equilibria Recursion Level 1 States Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Recursion Level 2 States Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Recursion Level 2 States, isolated Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions No Investment Recursion Level 2 States Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Multiple Equilibria Recursion Level 2 States Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Calculated Equilibria Recursion Level 2 States Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Continue recursion to calculate equilibria in all states Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Partial ordering of states Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Recursive Lexicographic Search Algorithm Building blocks of RLS algorithm: 1 State recursion algorithm solves the game conditional on equilibrium selection rule (ESR) 2 RLS algorithm e ffi ciently cycles through all feasible ESRs Properties of RLS algorithm: Complete: Computes all MPE equilibria of the game Fast: time spent of search of feasible ESRs is negligible in comparison to time spent on solving the game E ffi ciently skip infeasible ESRs Re-use results of previously computed subgames Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Represent ESR as equilibrium string of digits Use numbers in base- K number system with digits 0 , 1 , .., K − 1 Dependence preserving property: Any point of the state space may depend on points to the left and not the points to the right corner edges interior c e e e e i i i i c e e i c ESR string 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 0 0 0 0 0 0 0 0 1 1 1 1 2 c 0 0 0 2 1 2 2 1 1 1 1 2 2 2 c1 c2 0 2 1 0 0 2 1 2 1 1 2 1 2 2 End game Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions State space of the game: a “quarter pyramid” S = { ( c 1 , c 2 , c ) | c 1 ≥ c , c 2 ≥ c , c ∈ [0 , c ] } Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Particular ESRs examples c e e e e i i i i c e e i c ESR string 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 0 0 0 0 0 0 0 0 1 1 1 1 2 c c1 0 0 0 2 1 2 2 1 1 1 1 2 2 2 c2 0 2 1 0 0 2 1 2 1 1 2 1 2 2 End game 0 0 0 0 0 0 0 0 0 0 0 0 0 0 First equilibrium always Examples: 0 0 2 2 2 0 0 2 High cost to invest 2 2 0 0 0 2 2 0 Low cost to invest 1 1 1 1 1 1 Mixed when equal Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions All possible ESRs Lexicographic order c e e e e i i i i c e e i c ESR string 14 13 12 11 10 9 8 7 6 5 4 3 2 1 Lexicograph 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 2 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 2 1 4,782,969 0 0 0 0 0 0 0 0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 … 2 2 2 2 2 2 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Recalculation of fesibility condition for new ESR Avoid recalculation of subgames c e e e e i i i i c e e i c ESR string 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 always admissible Nr of eqb 1 1 1 1 1 3 3 3 3 1 1 1 3 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 1 0 admissible, solve 1 1 1 1 1 3 3 3 3 1 1 1 3 * No changes in the solution of the game Might have changed including the number of stage equilibria Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Jumping over blocks of infeasibles ESRs Using block structure of lexicographic ordering c e e e e i i i i c e e i c 1 Iteration: ESR string 14 13 12 11 10 9 8 7 6 5 4 3 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1a 1 1 1 1 1 3 3 3 3 1 1 1 3 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 1 0 2 1 2a 1 1 1 1 1 3 3 3 3 1 1 1 3 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 2 0 0 0 0 0 0 0 0 0 0 0 0 2 0 3 1 1 1 1 1 3 3 3 3 1 1 1 3 1 0 0 0 0 0 0 0 0 0 0 0 0 2 1 3a 0 0 0 0 0 0 0 0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 3b � 0 0 0 0 0 0 0 0 0 0 1 0 0 0 3c � 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3d � 0 0 0 0 0 0 0 0 1 0 0 0 0 0 4 � Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions RLS algorithm: running times K = 3 Simultaneous moves n = 3 n = 4 Total number ESRs 4,782,969 3,948,865,611 Number of feasible ESRs 127 46,707 Time used 0.008 sec. 0.334 sec. Simultaneous moves n = 5 Total number ESRs 174,449,211,009,120,166,087,753,728 Number of feasible ESRs 192,736,405 Time used 45 min. Alternating moves n = 5 Total number ESRs 174,449,211,009,120,166,087,753,728 Number of feasible ESRs 1 Time used 0.006 sec. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Another Road Map for the rest of Talk Part I: Results and Simulations from the Leapfroging Model 1 Resolution to the Bertrand investment paradox 2 Su ffi cient conditions for uniqueness of equilibria 3 Characterization of the set of equilibrium payo ff s 4 E ffi ciency of equilibria 5 Leap-frogging or preemption and rent-dissipation 6 Danger of imposing symmetry 7 Limitations of homotopy parameter methods 8 Implications for empirical work Part II: Dynamic Directional Games and RLS How to solve a general class of games using RLS Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Resolutions to Bertrand Investment Paradox Earlier work: Fudenberg et al. (1983 RIE), Reinganum (1985 QJE), Fudenberg and Tirole (1985 ReStud), .... Riordan and Salant (1994 JIE): Preemption and rent dissipation (unique equilibrium) We show: 1 Many types of endog. coordination is possible in equilibrium Leapfrogging (alternating investments) Preemption (investment by cost leader) Duplicative (simultaneous investments) 2 The equilibria are generally ine ffi cient due to over-investment Duplicative or excessively frequent investments Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Resolution to the Bertrand investment paradox Theorem (Solution to Bertrand investment paradox) If investment is socially optimal at a state point ( c 1 , c 2 , c ) 2 S , then no investment by both firms cannot be an MPE outcome in the subgame starting from ( c 1 , c 2 , c ) in either the simultaneous or alternating move versions of the dynamic game. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Multiplicity of equilibria Theorem (Su ffi cient conditions for uniqueness) In the dynamic Bertrand investment and pricing game a su ffi cient condition for the MPE to be unique is that 1 firms move in alternating fashion (i.e. m 6 = 0 ), and, 2 for each c > 0 in the support of ⇡ we have ⇡ ( c | c ) = 0 . 1 Corollary: If firms move simultaneously, equilibrium is generally not unique . 2 Corollary: If technological change is stochastic, equilibrium is generally not unique . Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Multiplicity of equilibria Theorem (Number of equilibria in simultaneous move game) If investment is socially optimal, and the support of the Markov process { c t } for the state of the art marginal costs is the full interval [0 , c 0 ] (i.e. continuous state version), the simultaneous move Bertrand investment and pricing game has a continuum of MPE. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Pay-o ff s in the simultaneous move game Theorem (Triangular payo ff s in the simultaneous move game) Suppose that the { c t } process has finite support, that there are no idiosyncratic shocks to investment (i.e. ⌘ = 0 ) and that firms move simultaneously The (convex hull of the) set of the expected discounted equilibrium payo ff s at the apex state ( c 0 , c 0 , c 0 ) 2 S is a triangle The vertices of this triangle are at the points (0 , 0) , (0 , V M ) and ( V M , 0) where V M = v N , i ( c 0 , c 0 , c 0 ) is the expected discounted payo ff to firm i in the monopoly equilibrium where firm i is the monopolist investor. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Pay-o ff s (deterministic tech progress, simultaneous moves) Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Pay-o ff s (stochastic tech progress, simultaneous moves) Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Pay-o ff s in the alternating move game Theorem (Equilibrium payo ff s in the alternating move game) The (convex hull of the) set of expected discounted equilibrium payo ff s at the apex state ( c 0 , c 0 , c 0 ) 2 S of the alternating game is a strict subset of the triangle with the vertices (0 , 0) , (0 , V M ) and ( V M , 0) Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Pay-o ff s: alternating vs simultaneous move games Panel (b): Simultaneous move Panel (a): Non − monotonic tech. progress 28528484 equilibria, 16510 distinct pay − off points 17826 equilibria, 792 distinct pay − off points Size: number of repetitions Color: efficiency Size: number of repetitions Color: efficiency 75.750 71.776 1 0.98 0.99 0.96 60 60 0.94 0.98 50 50 0.92 0.9 0.97 40 40 0.88 0.96 0.86 30 30 0.84 0.95 20 20 0.82 0.8 0.94 10 10 0.78 0.93 0 0 0 10 20 30 40 50 60 75.750 0 10 20 30 40 50 60 71.776 Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions E ffi ciency: alternating vs simultaneous move games Panel (c): Non − monotonic tech. progress Panel (d): Simultaneous move 8913 equilibria, 7817 leapfrog, 2752 mixed strategy 14264242 equilibria, 2040238 leapfrog, 2730910 mixed strategy 1 1 0.9 0.9 0.8 0.8 0.7 0.7 all equilibria all equilibria 0.6 0.6 leapfrog leapfrog mixed strategy mixed strategy 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 0.92 0.94 0.96 0.98 1 0.75 0.8 0.85 0.9 0.95 1 Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions E ffi ciency of equilibria Simultaneous move game Theorem (Ine ffi ciency of mixed strategy equilibria) A necessary condition for e ffi ciency in the dynamic Bertrand investment and pricing game is that along MPE path only pure strategy stage equilibria are played. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Riordan and Salant: Full Preemption Theorem ( Riordan and Salant, 1994) The continuous time investment game where 1 right to move alternates deterministically. 2 K ( c ) = K and is not prohibitively high. 3 technological progress is deterministic: c ( t ) is a continuous, decreasing function has a unique MPE with preemptive investments: by only one firm and no investment in equilibrium by its opponent. rent dissipation: discounted payo ff s of both firms in equilibrium is 0 , so the entire surplus is wasted on excessively frequent investments by the preempting firm. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Riordan and Salant: Full preemption and rent dissipation Confirm the result with high K and small dt Panel (a): Preemption and rent − dissipation 5 c 1 4.5 c 2 c monopoly 4 c Marginal Costs, Prices 3.5 3 2.5 2 1.5 1 0.5 0 0 10 20 30 40 Time Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Underinvestment Rent-dissapation is not a general outcome - disappears when K is low relative dt Panel (b): Underinvestment 5 c 1 4.5 c 2 c monopoly 4 c Marginal Costs, Prices 3.5 3 2.5 2 1.5 1 0.5 0 0 5 10 15 20 25 30 Time Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Leap-frogging Preemption is not the general outcome - disappears when K is even lower Panel (c): Leap − frogging 5 c 1 4.5 c 2 c monopoly 4 c Marginal Costs, Prices 3.5 3 2.5 2 1.5 1 0.5 0 0 5 10 15 20 25 30 Time Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Random alternation ! Leapfrogging Riordan and Salant’s result is not robust Panel (a): Random alternating moves 5 c 1 4.5 c 2 c monopoly 4 c Marginal Costs, Prices 3.5 3 2.5 2 1.5 1 0.5 0 0 5 10 15 20 25 30 Time Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Random onestep technology ! Leapfrogging Riordan and Salant’s result is not robust Panel (b): Non − monotonic tech. progress 5 c 1 c 2 4.5 c monopoly 4 c Marginal Costs, Prices 3.5 3 2.5 2 1.5 1 0.5 0 10 20 30 40 Time Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Random multistep technology ! Leapfrogging Riordan and Salant’s result is not robust Panel (c): Non − monotonic multistep tech. progress 5 c 1 4.5 c 2 c monopoly 4 c Marginal Costs, Prices 3.5 3 2.5 2 1.5 1 0.5 0 0 5 10 15 20 Time Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Simultaneous moves: Leapfrogging Riordan and Salant’s conjecture is wrong Panel (d): Simultaneous move 5 c 1 4.5 c 2 c monopoly 4 c Marginal Costs, Prices 3.5 3 2.5 2 1.5 1 0.5 0 0 5 10 15 20 25 30 Time Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Symmetric equilibria: V 1 ( c 1 , c 2 , c ) = V 2 ( c 2 , c 1 , c ) Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Limitations of homotopy approach Homotopy parameter: ⌘ In each period each firm incurs additive random costs/benefit from not investing and investing ⌘ is a scaling parameter that index variance of idiosyncratic shocks to investment High ⌘ ! unique equilibrium ⌘ ! 0 ! multiple equilibria Problems: Multiplicity of equilibria ! too many bifurcations along the path Equilibrium correspondence is not lower hemi-continuous Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Limits of the homotopy approach Equilibrium correspondance, alternating move game: V N , 1 ( c 0 , c 0 , c 0 ) vs. η Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Limits of the homotopy approach Video: Set of equilibrium outcomes as variance of shocks decreases to zero Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Sample objective criterion Identification and estimation problems in games with multiple equilibria Equilibria − specific objective functions Upper envelope Discontinuities Local maxima Global maxima L( θ ) θ θ 0 Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions Identification and estimation problems Problems in games with multiple equilibria Discontinuities in the objective function ! non-standard asymptotic inference. Kinks in objective function ! non-standard asymptotic inference. Multiple local optima ! leaves researchers clueless whether true ✓ is found. No unique maximum ! set identification ! standard asymptotic theory breaks down. As the number of equilibria grows, these problems increase. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions How to estimate θ ? Estimation strategies Brute force approach We can use RLS to compute all equilibria and construct upper envelope and do NFXP. MPEC Egesdal, Lai and Su (2012): MPEC, handles the non-smoothness. Two step methods ! small sample problems K-step stable NPL Aguirregabiria and Mira (2014): Nonparametric estimates of equilibrium strategies act as an equilibrium selection rule Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Recursive Lexicographic Search (RLS) Algorithm PART I : Solving the Leapfrogging Model with RLS Results and Simulations PART II: Recursive Lexicographical Search in General DDGs Limitations of homotopy approach Conclusions What now? Compare existing methods: We can generate data from leap-frogging model. Assuming a unique/multiple equilibrium at the true parameters. Assuming only one equilibrium played in the data, or one at each market (panel data). Investigate the identifying power of the Markov assumption, by comparing upper envelope likelihood for all MPE generated by RLS 1 likelihood derived from equilibrium sets generated by 2 equilibrium contraction methods. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions PART II The General Case: Recursive Lexicographical Search in Directional Dynamic Games Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Components of Markovian (stochastic) game G 1 n players who take actions at times t 2 { 1 , 2 , . . . , T } where T may be 1 , 2 A finite state space S and and action space A 3 state-specific constraint sets A i ( s ) representing the set of feasible actions of player i in state s 4 Von Neumann-Morgenstern utility functions u i ( s t , a t ) $ the payo ff to player i in state s t under actions a t 5 Markovian state transition probability p ( s 0 | s , a ), where a = ( a 1 , . . . , a n ) is the vector of actions chosen by players 6 Player-specific discount factors ( � 1 , . . . , � n ), � i 2 [0 , 1) 7 Common knowledge of state s and all the objects above 8 Private information ✏ i Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Behavior strategies � – feasible set of Markovian behavior strategies of the players in game G , i.e. n -tuple of mappings � = ( � 1 , . . . , � n ) where � i : S ! P ( A ) P ( A ) – set of all probability distributions on the set A . Feasibility requires that support( � i ( s )) ✓ A i ( s ) for each s 2 S A pure strategy is a special case where � i ( s ) places a unit mass on a single action a 2 A i ( s ) Σ ( G ) – set of all feasible Markovian strategies of the game G Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Definition of equilibium Definition (MPE) A Markov perfect equilibrium of the stochastic game G is a pair of feasible strategy n -tuple � ⇤ , and n -tuple of value functions V ( s ) = ( V 1 ( s ) , . . . , V n ( s )) where V i : S ! R , such that 1 the system of Bellman equations of the problem is satisfied (with the expectations taken probability distributions induced by opponents’ strategies in � ⇤ ), and 2 the strategies constitute mutual best responses of the players, and assign positive probabilities only to the actions in the set of maximizers of the Bellman equation. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Bellman equations ⇥ � u i ( s , ( a , � ⇤ V i ( s ) = max � i ( s )) + E a 2 A i ( s ) (X )# V i ( s 0 ) p ( s 0 | s , ( a , � ⇤ � i ( s ))) � i E , s 0 2 S where the expectation is taken over the probability distributions given by the opponents’ strategies � ⇤ j , j 6 = i . If the maximizer over a 2 A i ( s ) is unique, then � ⇤ i ( s ) is a unit mass on this optimal action. If there are multiple a 2 A i ( s ) that attain the maximum, then � ⇤ i ( s ) is a probability distribution whose support is a subset of the set of a 2 A i ( s ) that attain the maximum. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Bellman equations encode subgame perfection In definition of MPE the notion of “subgame perfectness” is reflected by the restriction implicit in the “Principle of optimality” of dynamic programming. For each player’s strategy � ⇤ i , the following holds Definition (Principle of Optimality/One Shot Deviation Principle) “whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the fir st decision” (Bellman, 1957). Thus the equilibrium is subgame perfect Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Directional components in the state space Suppose S can be written as S = D ⇥ X so s = ( d , x ) where we refer to d as the directional component of s and x as the non-directional component of s In the leapfrogging example, s = ( c 1 , c 2 , c , m ), so we have d = ( c 1 , c 2 , c ) and x = m . If G is a finite horizon game, then s = ( t , x ), where t denotes time. Then clearly d = t is the directional component of the state space. How to formalize the notion of directionality? Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Markovian Games as Directed Graphs (DGs) Games (a) to (c) are directional. What about game (d)? Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Strategy-specific partial order over D Let � be a feasible Markovian strategy in the game, and let ⇢ ( d 0 | ( d , x ) , � ( d , x )) be the conditional hitting probability of the state d 0 2 D starting from the state s = ( d , x ). Definition d 0 � σ d i ff ⇢ ( d 0 | ( d , x ) , � ( d , x )) > 0 and 9 x 2 X 8 x 0 2 X ⇢ ( d | ( d 0 , x 0 ) , ⇢ ( d 0 , x 0 )) = 0. Lemma � σ is a strict partial order of D , i.e. it is irreflexive, asymmetric, and transitive. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions The No-Loop (Anti-Cycling) Condition The partial order � σ defines a clear notion of directionality in the strategy-induced law of motion of the game G . Because � σ is transitive and asymmetric, there can be no loops (cycles) in any subset of comparable elements of D There are two ways ( d 0 , d ) can be non-comparable w.r.t � σ : a) there may be no communication between d 0 and d , b) there may be a loop (cycle) between d 0 and d . Definition: No Loop Condition d 0 6� σ d and d 0 6 = d i ff 8 x 2 X ⇢ ( d 0 | ( d , x ) , � ( d , x )) = 0 and 8 x 0 2 X ⇢ ( d | ( d 0 , x 0 ) , � ( d 0 , x 0 )) = 0 . Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Bargaining over a stochastically shrinking pie 0.4 0.4 0.2 1.0 0.8 0.5 d 1 d 2 d 3 d 4 0.6 0.1 Notice that d 2 and d 3 are not comparable under the induced partial order � σ . However the no-loop condition is satisified. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Bargaining over a shrinking/growing pie 0.4 0.4 0.2 1.0 d 1 0.3 d 2 d 3 d 4 0.6 0.5 0.6 This game induces the same partial order on D , � σ , but the no-loop condition fails due to the loop (cycle) between d 2 and d 3 . Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Consistency in Induced Partial Orders and DDGs Definition If � and � 0 are two feasible, Markovian strategies of G , we say the induced partial orders of D are consistent if d 0 � σ d = ) d 6� σ 0 d 0 Definition (DDG) A Dynamic Directional Game (DDG) is a finite state Markovian game G that satisfies the following two conditions: 1 Every feasible, Markovian strategy � satisifies the No-Loop Condition 2 Every pair of feasible, Markovian strategies � and � 0 induce consistent partial orderings, � σ and � σ 0 , respectively. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Defining a Strategy-Independent Partial Order for a DDG Definition Let � σ and � σ 0 be two strategy-induced partial orders of D . We say that � σ 0 is a refinement of � σ i ff 8 d , d 0 2 D we have d 0 � σ d = ) d 0 � σ 0 d Definition Let { � σ | � 2 Σ ( G ) } be a set of partial orders of D induced by the set of all feasible Markovian strategies in the game G , Σ ( G ). Then let � G be the join (or coarsest common refinement) of the the set of partial orders { � σ | � 2 Σ ( G ) } . Theorem The join partial order � G exists, is strategy-independent, and equals the transitive closure of the union of the partial orders in the set { � σ | � 2 Σ ( G ) } . Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DDGs and DAGs Definition A directed acyclic graph (DAG) is a directed graph with no cycles connecting any two vertices of the graph. Lemma If G is a finite state DDG, then the directed graph induced by � G , D ( G ), is a DAG. We use D ( G ) to partition D into T elements that are totally ordered and form the indices we need to do state recursion to find a MPE of G , D = { D 1 , D 2 , . . . , D T } . The ⌧ index is a generalization of the notion of a time index used in standard backward induction arguments. We say that ⌧ indexes the stage of the DDG G . Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DAG-recursion: identifying the stages of a finite state DDG Definition The terminal nodes of a DAG D ( G ) are the set of all vertices d 2 D that have no descendants. We let N ( D ( G )) denote the set of terminal nodes of D ( G ). Definition The non-terminal sub-DAG of a DAG D ( G ) is the DAG D 1 ( G ) given by D 1 ( G ) = D ( G ) � N ( D ( G )). Thus, D 1 ( G )) is the sub-DAG of D ( G ) the results when you remove its terminal nodes N ( D ( G )). Definition (DAG-Recursion) Define a sequence of sub-DAGs of D ( G ), {D τ ( G ) } recursively by D τ +1 ( G ) = D τ ( G ) � N ( D τ ( G )) . Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DAG-recursion terminates with the empty set Definition The stages of the finite state DDG G are given by { S 1 , S 2 , . . . , S τ , S τ +1 , . . . , S T } where S τ = ( D τ ⇥ X ) where { D 1 , . . . , D T } is the partition of D defined by the DAG-recursion. Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Finding the Stages of a Dynamic Directional Game A B C D E F G H I J K L M A DDG has a graph that is a DAG. Is this graph a DAG? Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions Recursion on Directed Acyclic Graphs (DAG-recursion) A B C D E F G H I J K L M Identify the terminal nodes of the DAG, S 1 = { G , M } Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DAG-recursion: eliminate terminal nodes A B C D E F H I J K L Now eliminate the terminal nodes to obtain the first “sub-DAG” Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DAG-recursion: identify the terminal nodes of the sub-DAG A B C D E F H I J K L Now identify the terminal nodes of the 1st sub-DAG, S 2 = { K } Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DAG-recursion: eliminate terminal nodes of sub-DAG 1 A B C D E F H I J L We now have identified stages S 1 = { G , M } , S 2 = { K } Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DAG-Recursion: identify terminal nodes of sub-DAG 2 A B C D E F H I J L Now identify the terminal nodes of the 2nd sub-DAG, S 3 = { L } Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DAG-recursion: eliminate I,J nodes to get sub-DAG 2 A B C D E F H I J We now have identified stages S 1 = { G , M } , S 2 = { K } , S 3 = { L } Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DAG-Recursion: eliminate E,H nodes to get sub-DAG 3 A B C D E F H S 1 = { G , M } , S 2 = { K } , S 3 = { L } , S 4 = { I , J } Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DAG-Recursion: eliminate D-node to get sub-DAG 4 A B C D F S 1 = { G , M } , S 2 = { K } , S 3 = { L } , S 4 = { I , J } , S 5 = { E , H } Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DAG-Recursion: eliminate C-node to get sub-DAG 5 A B C F S 1 = { G , M } , S 2 = { K } , S 3 = { L } , S 4 = { I , J } , S 5 = { E , H } , S 6 = { D } Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DAG-Recursion: eliminate B-node to get sub-DAG 6 A B F S 1 = { G , M } , S 2 = { K } , S 3 = { L } , S 4 = { I , J } , S 5 = { E , H } , S 6 = { D } , S 7 = { C } Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games

Introduction: Computing All Markov Perfect Equilibria Definition of Directional Dynamic Games (DDGs) PART I : Solving the Leapfrogging Model with RLS DAGs and State Recursion algorithm PART II: Recursive Lexicographical Search in General DDGs Recursive Lexicographical Search algorithm (RLS) Conclusions DAG-Recursion: eliminate A-node to get sub-DAG 7 A F S 1 = { G , M } , S 2 = { K } , S 3 = { L } , S 4 = { I , J } , S 5 = { E , H } , S 6 = { D } , S 7 = { C } , S 8 = { B } Iskhakov, Rust, and Schjerning (2013) Recursive Lexicographical Search in Dynamic Directional Games