Statistical physics of agent-based systems: Learning dynamics and complex co-operative behaviour in Minority Games Tobias Galla The Abdus Salam International Centre for Theoretical Physics and CNR/INFM SISSA Unit Trieste, Italy – p. 1/58
Acknowledgements Ginestra Bianconi (ICTP), Damien Challet (Oxford), Ton Coolen (London), Andrea De Martino (Rome), Matteo Marsili (ICTP), David Sherrington (Oxford), Yi-Cheng Zhang (Fribourg) support by European Community’s Human Potential Programme, Research Training Network STIPCO also EVERGROW and COMPLEX MARKETS – p. 2/58
Advertisement European Conference on Complex Systems Saïd Business School, Oxford UK, 25-29 September 2006 Satellite workshop on ’Complex Adaptive Systems and Interacting Agents’ organised by Andrea De Martino, Enzo Marinari, David Sherrington and myself http://chimera.roma1.infn.it/CASIA/ – p. 3/58
Minority Game ... originally a simple model for inductive decision making of agents (El-Farol bar problem) Interest by economists simple model of a market, stylised facts ... physicists phase transitions, ergodicity breaking, spin glass problem, off-equilibrium dynamics mathematicians exact solutions – p. 4/58
Stock market traders particles, spins, microscopic degrees of freedom they observe a price time-series (and other information) externally and/or internally generated information based on this they buy/sell interaction price is formed based on their actions macroscopic observable, mean-field they learn and adapt (some better than others maybe) dynamics – p. 5/58
Stock market traders particles, spins, microscopic degrees of freedom they observe a price time-series (and other information) externally and/or internally generated information, history, can be non-Markovian based on this they buy/sell decision making (noise ...) price is formed based on their actions global interaction, macroscopic observable, mean-field they learn and adapt (some better than others maybe) dynamics, update rules, equations of motion – p. 6/58
The model: Minority Game [Challet, Zhang 1997] N traders i = 1 , . . . , N given signal µ ( t ) ∈ { 1 , . . . , P } at each time-step here: random external information then every player has to make a binary trading decision b i ( t ) ∈ {− 1 , 1 } all players in minority are successful, players in majority unsuccessful if A ( t ) is the total bid A ( t ) = � i b i ( t ) , then payoff for i is − b i ( t ) A ( t ) – p. 7/58
The model: Minority Game How do players make trading decisions ? everybody has S trading strategies � a i,s , s = 1 , . . . , S mapping µ onto a µ i ∈ {− 1 , 1 } (buy or sell) Strategy is a table mapping µ onto binary decision 1 2 3 4 ... P µ a µ -1 1 1 -1 ... -1 i Given history µ a strategy table tells me to play a µ i . – p. 8/58
The model: Minority Game Consider case S = 2 strategies per player in the following µ 1 2 3 4 ... P strategy s = +1 a µ -1 1 1 -1 ... -1 i,s =+1 µ 1 2 3 4 ... P strategy s = − 1 a µ 1 1 -1 -1 ... 1 i,s = − 1 Then what this player has to decide at time t is which of the two tables to use. Assign scores to each strategy to measure their success. – p. 9/58
The model: Minority Game aim: to be in the minority which strategy to use ? The one which has performed best so far ! to assess performance keep a score for each strategy: ( − a µ ( t ) u i,s ( t + 1) = u i,s ( t ) + i,s A ( t )) � �� � minority game payoff strategies generated randomly before start of the game – p. 10/58
MG dynamics [Marsili’s slide] – p. 11/58
MG for physicists – p. 12/58
The phenomenology of the basic MG What are the interesting observables ? And what are the model parameters ? – p. 13/58
The phenomenology of the basic MG Model parameters ... just one. α = number of values information can take = P number of agents N i.e. α high: large information space and/or small market low α means the opposite: large market and/or small information space – p. 14/58
Observables Predictability � P µ =1 � A | µ � 2 H = 1 P H > 0 ⇒ � A | µ � � = 0 statistically predictable H = 0 ⇒ � A | µ � = 0 predictability zero global performance/volatility � A 2 � σ 2 = = − total gain [Challet, Marsili, Zecchina] – p. 15/58
Observables Predictability � P µ =1 � A | µ � 2 H = 1 P H > 0 ⇒ � A | µ � � = 0 statistically predictable H = 0 ⇒ � A | µ � = 0 predictability zero global performance/volatility ✻ � A 2 � σ 2 = = − total gain Phase transition between a predictable and an unpredictable phase – p. 16/58
Ergodicity breaking tabula rasa start biased start 2 σ ergodic 1 no memory non-ergodic, memory 0 −1 0 1 10 10 10 α – p. 17/58
Ergodicity breaking tabula rasa start biased start 2 σ ergodic 1 no memory non-ergodic, memory 0 −1 0 1 ✻ 10 10 10 α Phase transition between a non-ergodic and an ergodic phase – p. 18/58
Ergodicity breaking static susceptibilities of CuMn, field-cooling versus zero-field cooling – p. 19/58
MG as spin glass model MG shares many features with spin-glass models ij = 1 � J 2 H SK = J ij s i s j , N ij [Sherrington-Kirkpatrick model, SK 1975] frustration (not everybody can win) quenched disorder (random strategy assignments) mean-field interactions (interaction with ev’body else) – p. 20/58
Remember S = 2 strategies per player: 1 2 3 4 ... P µ s i = +1 , score u i + a µ -1 1 1 -1 ... -1 i,s =+1 1 2 3 4 ... P µ s i = − 1 , score u i − a µ 1 1 -1 -1 ... 1 i,s = − 1 Then what this player has to decide at time t is which of the two tables to use. s i ( t ) = sgn [ u i + ( t ) − u i − ( t )] – p. 21/58
Learning dynamics proposed action total action ❍❍ ✟ ✟ ❥ ❍ ✙ ✟ u i, + ( t + 1) = u i, + ( t ) − a µ ( t ) i, + A ( t ) u i, − ( t + 1) = u i, − ( t ) − a µ ( t ) i, − A ( t ) Evolution of score difference ( q i = u i + − u i − ): � � a µ ( t ) i, + − a µ ( t ) q i ( t + 1) = q i ( t ) − A ( t ) i, − – p. 22/58
Learning dynamics On-line update for score difference ( q = u + − u − ): � � a µ ( t ) i, + − a µ ( t ) q i ( t + 1) = q i ( t ) − A ( t ) i, − and � A ( t ) = f ( sgn [ q j ( t )] | strategies of j ) j Batch update for score difference (average over µ ): � q i ( t + 1) = q i ( t ) − J ij sgn [ q j ( t )] − h i j quenched disorder, spin glass problem ( a µ j + − a µ ( a µ j + + a µ ( a µ i + − a µ ( a µ i + − a µ P N P X X X i − ) j − ) i − ) j − ) J ij = 1 h i = 1 , P 2 2 P 2 2 µ =1 j =1 µ =1 | {z } Hebbian – p. 23/58
Dynamics � q i ( t + 1) − q i ( t ) = − J ij sgn [ q j ( t )] − h i i but not J ij q j ( t ) − h i = − ∂H [ q ] � q i ( t + 1) − q i ( t ) = − ∂q i i No gradient-descent. No detailed balance. Still pseudo-Hamiltonian: H ( s ) = 1 � � J ij s i s j + h i s i 2 ij i – p. 24/58
MG as an anti-Hopfield model H ( s ) = 1 1 � � � ξ µ i ξ µ J ij s i s j + J ij = h i s i , j 2 αN µ ij i Hopfield model has H ( s ) = − 1 � J ij s i s j 2 ij MG is an ‘unlearning’ game. – p. 25/58
MG for mathematicians – p. 26/58
Generating functional analysis [Heimel, Coolen PRE 2001] � J ij sgn [ q j ( t )] − h i + q i ( t + 1) − q i ( t ) = − ϑ ( t ) ���� j perturbationfield Dynamical partition function ! Z X Z [ ψ ] = D q δ ( eq of motion ) exp i ψ i ( t ) sgn [ q i ( t )] it 0 1 Z @X X A = D q D b q exp q i ( t )[ q i ( t + 1) − q i ( t ) + b J ij sgn [ q j ( t )] + h i − ϑ ( t )] it j ! X × exp i ψ i ( t ) sgn [ q i ( t )] it Then path integrals, disorder-average, saddle-point equations ... – p. 27/58
Generating functional analysis tt ′ sgn [ q ( t ′ )] + √ αη ( t ) � [ I + G ] − 1 q ( t + 1) = q ( t ) + ϑ ( t ) − α t ′ with noise covariance [( I + G ) − 1 D ( I + G T ) − 1 ] tt ′ < η ( t ) η ( t ′ ) > = = 1 + C tt ′ D tt ′ Dynamical order parameters: ∂ C tt ′ = < sgn [ q ( t )] sgn [ q ( t ′ )] >, G tt ′ = ∂ϑ ( t ′ ) < sgn [ q ( t )] > [Heimel/Coolen PRE 2001] [Coolen/Heimel J Phys A 2001] [Coolen J Phys A 2005] – p. 28/58
Basic MG 1 1 10 q(0)=0.1 q(0)=0 q(0)=0.5 q(0)=1 q(0)=1 q(0)=10 0.8 q(0)=10 0.6 σ c 0 10 0.4 0.2 −1 10 0 −1 0 1 2 −1 0 1 2 10 10 10 10 10 10 10 10 α α EA parameter volatility exact result [Heimel/Coolen] approximation: drop transients [Heimel/Coolen] – p. 29/58
Spherical MG Replace � q i ( t + 1) − q i ( t ) = − J ij sgn [ q j ( t )] − h i � �� � i Ising by � q i ( t + 1) − q i ( t ) = − φ j ( t ) − h i , J ij � �� � j continuous with φ i = q i � φ 2 i = N λ , i [Galla, Coolen, Sherrington J Phys A 2003] [Galla, Sherrington JSTAT 2005] – p. 30/58
Spherical MG Z = � � φ 2 − N ) exp( − βH ) d� φ δ ( � { s i = ± 1 } exp( − βH ) Z = [Kac, Berlin ‘The Spherical Model of a Ferromagnet’, Phys. Rev. 86, 821-835 (1952)] – p. 31/58
Recommend
More recommend
