Overview of the Simulation Modeling Process Nathaniel Osgood - PowerPoint PPT Presentation

Overview of the Simulation Modeling Process Nathaniel Osgood Agent-Based Modeling Bootcamp for Health Researchers August 22, 2011

Overview of Modeling Process • Typically conducted with an interdisciplinary team • An ongoing process of refinement • Best: Iteration with modeling, intervention implementation, data collection • Often it is the modeling process itself – rather than the models created – that offers the greatest value

ABM Modeling Process Overview A Key Deliverable! Learning Reference mode Specification & Parameter sensitivity environm Specification of reproduction investigation of analysis Model scope/boundary ents/Mic Causal loop diagrams intervention scenarios • Parameters Matching of selection. roworlds Cross-validation Investigation of State charts intermediate time /flight • Quantitative causal Model time horizon hypothetical external Robustness&extreme series simulator System Structure relations conditions value tests diagrams s Matching of • Decision/behavior Identification of key Cross-scenario Unit checking observed data points Multi-agent interaction rules comparisons (e.g. CEA) variables Problem domain tests diagrams Constrain to sensible • Initial conditions Reference modes for bounds Multi-scale hierarchy explanation Structural sensitivity diagrams analysis Process flow structure Group model building

Recall: Coevolution Observations/ Evaluation External World Mental Model Actions & Choice of Observations Simulated Dynamics Formal Modeling Artifacts

Modeling Process Overview A Key Deliverable! Learning Reference mode Specification & Parameter sensitivity environm Specification of reproduction investigation of analysis Model scope/boundary ents/Mic Causal loop diagrams intervention scenarios • Parameters Matching of selection. roworlds Cross-validation Investigation of State charts intermediate time /flight • Quantitative causal Model time horizon hypothetical external Robustness&extreme series simulator System Structure relations conditions Identification of value tests diagrams s Matching of • Decision/behavior Key variables Cross-scenario Unit checking observed data points Multi-agent interaction rules comparisons (e.g. CEA) Problem domain tests Reference modes for diagrams Constrain to sensible • Initial conditions explanation bounds Multi-scale hierarchy Structural sensitivity diagrams analysis Process flow structure Group model building

Identification of Questions/ “The Problem” • All models are simplifications and “wrong” • Some models are useful • Attempts at perfect representation of “real - world” system generally offer little value • Establishing a clear model purpose is critical for defining what is included in a model – Understanding broad trends/insight? – Understanding policy impacts? – Ruling out certain hypotheses? • Think explicitly about model boundaries • Adding factors often does not yield greater insight – Often simplest models give greatest insight – Opportunity costs: More complex model takes more time to build=>less time for insight

Importance of Purpose Firmness of purpose is one of the most necessary sinews of character, and one of the best instruments of success. Without it genius wastes its efforts in a maze of inconsistencies. Lord Chesterfield The secret of success is constancy of purpose. Benjamin Disraeli The art of model building is knowing what to cut out, and the purpose of the model acts as the logical knife. It provides the criterion about what will be cut, so that only the essential features necessary to fulfill the purpose are left. John Sterman H Taylor, 2001

Common Division • Endogenous – Things whose dynamics are calculated as part of the model • Exogenous – Things that are included in model consideration, but are specified externally • Time series • Constants • Ignored/Excluded – Things outside the boundary of the model

Example of Boundary Definition (1998)

Modeling Process Overview A Key Deliverable! Learning Reference mode Specification & Parameter sensitivity environm Specification of reproduction investigation of analysis Model scope/boundary ents/Mic Causal loop diagrams intervention scenarios • Parameters Matching of selection. roworlds Cross-validation Investigation of State charts intermediate time /flight • Quantitative causal Model time horizon hypothetical external Robustness&extreme series simulator System Structure relations conditions Identification of value tests diagrams s Matching of • Decision/behavior Key variables Cross-scenario Unit checking observed data points Multi-agent interaction rules comparisons (e.g. CEA) Problem domain tests Reference modes for diagrams Constrain to sensible • Initial conditions explanation bounds Multi-scale hierarchy Structural sensitivity diagrams analysis Process flow structure Group model building

Example Causal Loop Diagram Department of Computer Science

A Second Causal Loop Diagram Susceptibles - + + New Infections Contacts of Susceptibles with Infectives + + Infectives + People Presenting for Treatment + Waiting Times - Health Care Staff

Qualitative Causal Loop Diagram Qualitative Transitions (no likelihood yet specified)

These variables are aspects of state . Weight Cumulative Cigarettes Smoked Age These “parameters” give static These describe the “ behaviours ” – the mechanisms that will characteristics of the agent govern agent dynamics

Stock & Flow Structure <Birth Rate> Pregnant Normal Normal and Pregnancies to Weight Mothers Non-Overweight Mother Underweight with No GDM Pregnancies of Normal Weight Developing GDM Weight Non-Overweight History Deaths Women Completion of Pregnancy to Non-Overweight State Pregnancy Duration Pregnant Women Developing Persistent Shedding Obesity Overweight/Obesity Developing Obesity Pregnant with GDM Normal Weight Individuals Developing Pregnancies Developing T2DM GDM from Mother with GDM History Pregnant Pregnancies to Overweight Overweight Mother Developing GDM erweight Babies Born to Mothers with No Completion of regnant Normal Weight Overweight Pregnancy to GDM History Mothers Overweight State Completion of GDM Pregnancy Pregnancies of Oveweight Babies Born bies Born from Overweight Overweight Women from T2DM Mothers other with Deaths GDM Pregnant Women with GDM that Continue on to Overweight Individuals Developing T2DM Postpartum T2DM Pregnancies for Pregnant with Women with GDM T2DM Pre-Existing History of History of GDM Women with History of T2DM Deaths GDM GDM Developing T2DM New Pregnancies from Completion of Non-GDM Completion of Pregnancy for Mother with T2DM Mother with T2DM Pregnancy for Woman with History of GDM Pregnant with T2DM Deaths from Non-T2DM Women with History of GDM

Problem Mapping: Qualitative Models (System Structure Diagram) Headley, J., Rockweiler, H., Jogee, A. 2008. Women with HIV/AIDS in Malawi: The Impact of Antiretroviral Therapy on Economic Welfare, Proceedings of the 2008 International Conference of the System Dynamics Society, Athens, Greece, July 2008.

Modeling Process Overview A Key Deliverable! Learning Reference mode Specification & Parameter sensitivity environm Specification of reproduction investigation of analysis Model scope/boundary ents/Mic Causal loop diagrams intervention scenarios • Parameters Matching of selection. roworlds Cross-validation Investigation of State charts intermediate time /flight • Quantitative causal Model time horizon hypothetical external Robustness&extreme series simulator System Structure relations conditions Identification of value tests diagrams s Matching of • Decision/behavior Key variables Cross-scenario Unit checking observed data points Multi-agent interaction rules comparisons (e.g. CEA) Problem domain tests Reference modes for diagrams Constrain to sensible • Initial conditions explanation bounds Multi-scale hierarchy Structural sensitivity diagrams analysis Process flow structure Group model building

Model Formulation • Model formulation elaborates on problem mapping to yield a quantitative model • Key missing ingredients – Specifying formulas for • Statechart transitions • Flows (in terms of other variables) • Intermediate/output variables – Parameter values

Example Conditional Transition The incoming transition into “ WhetherPrimaryProgre ssion ” will be routed to thisoutgoing transition if this condition is true

Transition Type: Message Triggered

Transition Type: Fixed Rate

Transition Type: Variable Rate

Transition Type: Fixed Residence Time (Timeout)

Simple Intermediate Variable

Sources for Parameter Estimates • Surveillance data • Controlled trials • Outbreak data • Clinical reports data • Intervention outcomes studies • Calibration to historic data • Expert judgement Anderson & May • Systematic reviews