The Game of Life, Decision & Communication Roland M uhlenbernd - PowerPoint PPT Presentation

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION The Game of Life, Decision & Communication Roland M¨ uhlenbernd

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION O VERVIEW 1. Introduction: The Game Of Life 2. Pre-Decision 3. Learning 4. Communication

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION G AME OF L IFE ’ S R ULES OF N ATURE 1. under-population: any alive cell with fewer then two alive X du neighbor cells dies 2. surviving: any alive cell with two or three alive neighbor cells lives X do on to the next generation 3. overcrowding: any alive cell with X s X s X s more than three alive neighbor cells dies 4. reproduction: any dead cell with X r exactly three alive neighbors becomes an alive cell

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION G AME OF L IFE ’ S R ULES OF N ATURE Play the Game of Life on http://www.bitstorm.org/gameoflife/ or http://www.denkoffen.de/Games/SpieldesLebens/

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION D ECREASING O CCUPATION S HARE Basic Game of Life 1600 1400 1200 1000 800 600 400 200 0 0 500 1000 1500 2000 2500 3000 Figure : The number of alive cells decreases from initially around 1225 (25 % ) to finally 158 (3 . 2 % ) on average over 15 runs (70x70 grid).

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION T HE NON - DETERMINISTIC n - DIE GAME P1: Initialization 1. Create a list of all alive cells in a random order P2: Sacrifice Decision 2. Delete successively all cells with n neighbors P3: Rules of Nature 3. Apply the rules of nature of the game of life

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION 3-die modification 4-die modification 1400 1600 1400 1200 1200 1000 1000 800 800 600 600 400 400 200 200 0 0 0 500 1000 1500 2000 2500 3000 0 500 1000 1500 2000 2500 3000 3-die game (1.8%) 4-die game (6.9%) 5-die modification 6-die modification 1600 1600 1400 1400 1200 1200 1000 1000 800 800 600 600 400 400 200 200 0 0 500 1000 1500 2000 2500 3000 0 500 1000 1500 2000 2500 3000 5-die game (14.7%) 6-die game (3%)

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION 0.15 0.10 0.05 basic game.3.die game.4.die game.5.die game.6.die

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION F ROM S ITUATIONS TO A CTIONS ◮ Set of states T = { t 1 , t 2 , t 3 , t 4 , t 5 , t 6 , t 7 , t 8 } ◮ Set of situations Γ = { γ = � t i , t j �| t i ∈ T is the state of an alive cell c , t j ∈ T the state of an alive neighbor cell of c } ◮ Set of actions A = { a die , a stay } X X � t 2 , t 2 � � t 2 , t 3 � X � t 3 , t 1 � X � t 1 , t 3 �

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION R EINFORCEMENT L EARNING Reinforcement learning account RL = { σ, Ω } ◮ response rule σ ∈ (Γ → ∆( A )) ◮ update rule Ω : if action a is successful in situation γ , then increase the probability σ ( a | γ ) ◮ an action a is considered as successful, if and only if OS a > OS ¬ a

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION T HE n × m -D IE L EARNING G AME P1: Initialization 1. Initialize an RL account for Γ and A P2: Sacrifice Decision 2. For all c i ∈ C : 2.1 pick randomly a neighbor c j ∈ N i and request its state t m 2.2 play action a via response rule σ ( a |� t n , t m � ) , where t n is the state of c i 2.3 if a = a die delete cell c i , RL update Ω P3: Rules of Nature 3. Apply the rules of nature of the game of life

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION R ESULTS Signaling with given meaning 2500 0.35 0.30 2000 0.25 1500 0.20 0.15 1000 0.10 500 0.05 0.00 0 0 500 1000 1500 2000 2500 3000 all successful failed Course of 20 simulation runs Box plots of all, successful and failed runs ◮ the average occupation share over all runs is 17 . 6 % (862 cells) ◮ the average occupation share of successful runs is 28 . 4 % (1392 cells), for failed runs 1 . 4 % (69 cells)

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION R ESULTS Definition (Neighbor treatment rules) For the n × m-die learning game a successful strategy can be characterized by the following two rules: 1. Sacrifice if your neighbor has exactly 4 neighbors. 2. Never sacrifice if your neighbor has less than 4 neighbors.

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION B UT ... ”In our opinion, the property of access restriction to direct neighborhood information is an important requirement for all following pre-games since this property reflects the spatial character of the rules of nature of the game of life. We denote this requirement as the local information rule .” X X � t 2 , t 2 � � t 2 , t 3 � X � t 3 , t 2 � X � t 1 , t 3 �

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION S IGNALING G AMES A signaling game SG = � ( S , R ) , T , M , A , U � is ◮ played between a sender S and a receiver R ◮ S has private information state t ∈ T ◮ S sends a message m ∈ M ◮ R responds with a choice of action a ∈ A ◮ U : T × A → R defines the success of communication

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION T HE n - MESSAGES SIGNALING GAME P1: Initialization 1. Create a RL account for the signaling game SG n = � ( S , R ) , T , M , A , U � witn n messages P2: Sacrifice Decision 2. For all c i ∈ C : 2.1 pick randomly a neighbor c j ∈ N i and make a state request for its state t 2.2 c j sends a message m ∈ M via response rule σ ( m | t ) 2.3 c i plays action a ∈ A via response rule σ ( a | m ) 2.4 if a = a die delete cell c i , RL-update Ω P3: Rules on Nature 3. Apply the rules of nature of the game of life

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION R ESULTING S UCCESSFUL S TRATEGIES t 1 t 1 m a a die t 2 t 2 a stay m a a die t 3 t 3 m b t 4 t 4 m c t 5 t 5 a stay m b t 6 t 6 m d t 7 t 7 t 8 t 8 Result for 2 messages Result for 4 messages

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION R ESULTS ◮ Always one ”death message”, but often multiple ”survive messages” and unused messages ◮ Successful strategies realize ”Neighbor treatment rules” ◮ Strong tendency for � t 5 , a stay � and � t 6 , a die � ◮ The rate for learning a successful strategy increases with the number of messages percentage of runs 1 0.8 0.6 0.4 0.2 0 n = 2 4 6 8

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION O UTLOOK ◮ How do rules of nature affect evolving signaling systems? → Experiments with changed rules of nature ◮ General question: how do signaling strategies evolve under selective pressure determined by environmental / nature rules?

T HE G AME O F L IFE P RE -D ECISION L EARNING C OMMUNICATION Thanks for attention!