distributing Agent- based simulations Gennaro Cordasco, Rosario De - PowerPoint PPT Presentation

A Framework for distributing Agent- based simulations Gennaro Cordasco, Rosario De Chiara, Ada Mancuso, Dario Mazzeo, Vittorio Scarano and Carmine Spagnuolo

Outline • Agent Based Simulations (ABMs) • MASON • Distributed ABMs • DMASON – issues – architecture • MASON vs DMASON • Tests 8/29/2011 HeteroPar'2011 2

Agent-based simulation • Simulate the actions and interactions of autonomous individual agents with a view to assessing their effects on the system as a whole – Investigated since the 1980s – Early works take inspiration from particles [R83] • Applications – Biology, economics, sociology … • Why parallel? – Huge number of agents – Complex behaviours 8/29/2011 HeteroPar'2011 3

Motivation Social Science Economy Science Biology Fisics Artificial Intelligence 4

An Example : Reynolds’ Model • Alignment : steer towards the average heading of local flock-mates • Cohesion : steer to move toward the average position of local flock-mates • Separation : steer to avoid crowding local flock- mates Alignment Cohesion Separation Alignment Cohesion Separation

Features of AB Simulation platforms • Flexibility – customization of agents and their behaviors • Speed and efficiency – scalable beyond millions, hundred of millions, billions – batch processing – interactive simulation rate (in the order of seconds) • Testing and validation suites • Research facilities (logging, graphing, etc.) • Open-source – communities of public/private entities, users, developers, researchers are crucial to ensure long-term sustainability and deep impact on current research practices 8/29/2011 HeteroPar'2011 6

Massive Agent-Based Simulations 7 • Massive : when the simulated agents are – extremely numerous, – complex to simulate – with non-linear, dynamic behavior • A Short Answer: "smart" parallelization 8/29/2011 HeteroPar'2011

Why MASON ? • Mason is recognized to be expressive and efficient • Mason structure: clear separation between Simulation and Visualization • Back compatibility with simulations already present in the framework 8/29/2011 HeteroPar'2011 8

DMASON Field Master Agents Regions Workers Communication Server 8/29/2011 HeteroPar'2011 9

DMASON Issues Work Partitioning Synchronization Communication Reproducibility

Agents vs Space Partitioning • Agents Partitioning assigns a fixed number of agents to each available worker – Self balanced – Requires an all to all communication • Space Partitioning partitions the simulation space into regions. Each region is assigned to a worker which is in charge of simulating all the agents belonging to the region – A small amount of communication is required – Agents can migrate – Load balancing is not guaranteed 8/29/2011 HeteroPar'2011 11

DMASON: Field Partitioning 8/29/2011 HeteroPar'2011 12

DMASON: Field Partitioning 1 2 3 4 8/29/2011 HeteroPar'2011 13

DMASON: Field Partitioning • DMASON allows to partition the field into regions • Neighboring regions communicate before each simulation step MASON DMASON 1D DMASON 2D HeteroPar'2011 14

DMASON: Field Partitioning • A portion of each Right Region region is exchanged between neighbor workers before each simulation step • The size of this portion depends on agents area of Left Region interest (AOI) 8/29/2011 HeteroPar'2011 15

DMASON: Synchronization • Local synchronization: – The step i of region r is computed by using the states i − 1 of r’s neighborhood – The step i of a region cannot be i i-1 i-1 executed until the states i − 1 of its neighborhood have been computed i i i-1 and delivered. • No central coordinator i i i • Simulation speed  slowest region speed 8/29/2011 HeteroPar'2011 16

DMASON: Communication • DMASON uses the publish – subscribe design pattern to propagate agents state information – Current version of DMASON uses Java Message Service (JMS) for communication between workers

DMASON: Reproducibility • Agents evolve simultaneously – each simulation step can be Step i Step i+1 executed in parallel overall the agents – the order in which agents are scheduled does not affect the reproducibility of tmp buffer results – neighbors’ updates are always processed in the same order 8/29/2011 HeteroPar'2011 18

DMASON Architecture • DMASON is a new layer which extends the MASON simulation layer. • The new layer does not alter in any way existing layers 8/29/2011 HeteroPar'2011 19

DMASON Architecture The same structure as MASON Distributed State Distributed dmason.Engine Schedule Remote Agent<E> DMASON DContinuous2D DSparseGrig2D Distributed dmason.Field DDoubleGrid2D Field Connection DIntGrid2D Management DObjectGrid2D dmason.Util Exception Resources 8/29/2011 HeteroPar'2011 20

MASON vs DMASON DFlockers

DMASON: System Management Worker Master Console 8/29/2011 HeteroPar'2011 22

Testing DMASON • We want to assess that DMASON… – …is able to run simulations that are impractical or impossible to execute with MASON – …is scalable – …allows to develop reproducible simulations (independently from the number of workers) – …allows to exploit multicore CPU – …allows to exploit heterogeneous hardware 8/29/2011 HeteroPar'2011 23

DMASON: homogeneous hosts 7 HOST : Intel i7-2600 4x3,4GHz con HT 8GB RAM JVM 32 bit 14 MASON DMASON 7 Hosts DMASON 6 Hosts Avg simulation step timing (s) 12 DMASON 4 Hosts DMASON 5 Hosts 10 8 Mason 6 DMASON 2x2, (1,1,1,1) 4 DMASON 3x3, (2,2,2,2,1) DMASON 4x4, (3,3,3,3,2,2) 2 DMASON 5x5, (4,4,4,4,3,3,3) 0 0 2000000 4000000 6000000 8000000 10000000 12000000 14000000 Agents 8/29/2011 HeteroPar'2011 24

DMASON: heterogeneous hosts 8/29/2011 HeteroPar'2011 25

DMASON: heterogeneous hosts 3,000,000 Agents 19,65 20 5,000,000 Agents Avg simulation step timing (s) 15 12,70 11,87 11,48 10,75 9,88 9,15 9,16 10 7,43 7,24 6,25 5,65 5,61 5,52 5 0 (0,0,0,0,25) (1,1,1,1,21) (1,2,2,2,18) (1,2,2,3,17) (1,2,2,4,16) (1,3,3,6,12) (5,5,5,5,5) Configurations (P4, Xeon, Opt, I5,I7)

Conclusion • ABMs are CPU intensive applications and requires large amount of memory • The need for complex simulations involving a large number of agents is always felt by researchers and practitioners • DMASON allows to achieve those requirements by harvesting the unused CPU power and Memory 8/29/2011 HeteroPar'2011 27

Current work • Simulation logging and replay Field Master Agents Regions Workers Logging Server Communication Server Logger Visualizers

Current work • Visualization Field Master Agents Regions Workers Logging Server Communication Server Logger Visualizers

Future Development • Development of other distributed fields • Dynamic Load Balancing • Distributed communication 8/29/2011 HeteroPar'2011 30

Thanks for your attention Gennaro Cordasco, Rosario De Chiara, Ada Mancuso, Dario Mazzeo, Vittorio Scarano and Carmine Spagnuolo http://www.isislab.it/projects/dmason/