On Algorithmic Decision Procedures in Emergency Response Systems in - PowerPoint PPT Presentation

On Algorithmic Decision Procedures in Emergency Response Systems in Smart and Connected Communities Geoffrey Pettet 1 , Ayan Mukhopadhay 2 , Mykel Kochenderfer 2 , Yevgeniy Voroybeychik 3 , Abhishek Dubey 1 1 Vanderbilt University, 2 Stanford University, 3 Washington University in St Louis Sponsored by National Science Foundation, Center for Automotive Research at Stanford (CARS), and Tennessee Department of Transportation Institute for Software Integrated Systems World-class, interdisciplinary research with global impact.

Motivation and Background Institute for Software Integrated Systems 1 World-class, interdisciplinary research with global impact.

The emergency response problem All traffic incidents occurring in Davidson County In January 2018, with a sliding window of ~12 hours The problem: Respond Efficiently to all incidents spread over a large geographic area with limited resources. Institute for Software Integrated Systems 2 World-class, interdisciplinary research with global impact.

Proactive Emergency Response Active Learning and Improvement Mechanisms Online Demand Anticipatory Optimal Estimation Stationing of Dispatch [1] Models [1] Resources [1] Ayan Mukhopadhyay, Geoffrey Pettet, Chinmaya Samal, Abhishek Dubey, and Yevgeniy Vorobeychik. 2019. An online decision- theoretic pipeline for responder dispatch. In Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS ’19). Association for Computing Machinery, New York, NY, USA, 185–196. DOI:https://doi.org/10.1145/3302509.3311055 Institute for Software Integrated Systems 3 World-class, interdisciplinary research with global impact.

Proactive Emergency Response Active Learning and Improvement Mechanisms Online Demand Anticipatory Optimal Estimation Stationing of Dispatch [1] Models [1] Resources Previous [1] Work Prediction Actual Tree Search (MCTS) [1] Ayan Mukhopadhyay, Geoffrey Pettet, Chinmaya Samal, Abhishek Dubey, and Yevgeniy Vorobeychik. 2019. An online decision- theoretic pipeline for responder dispatch. In Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS ’19). Association for Computing Machinery, New York, NY, USA, 185–196. DOI:https://doi.org/10.1145/3302509.3311055 Institute for Software Integrated Systems 4 World-class, interdisciplinary research with global impact.

Proactive Emergency Response Active Learning and Improvement Mechanisms Online Demand Anticipatory Optimal Estimation Stationing of Dispatch [1] Models [1] Resources Focus of Paper “Rebalancing” Advantages over decision making at time of dispatch: • Ample time to make decision • Avoids legal and moral questions • Proactive • Larger decision space => more room for gains Institute for Software Integrated Systems 5 World-class, interdisciplinary research with global impact.

System Model and Assumptions Institute for Software Integrated Systems 6 World-class, interdisciplinary research with global impact.

System em Model el – Assu Assumption ions Historical data (incidents, traffic, weather, etc.) https://www.nashville.gov Region “Depots” – segmented into Incident subset of cells a grid with arrival model where agents equally sized can wait cells Institute for Software Integrated Systems 7 World-class, interdisciplinary research with global impact.

System Model – Multi Agent SMDP I c : Grids waiting for service • R c : Agent states I c+1 , R c+1 , E c+1 I c+2 , R c+2 , E c+2 • σ c+2 σ c-1 σ c+1 Action set E c : Environmental Factors • σ c s c-1 s c+1 s c+2 Current State s c *SMDP diagram simplified for demonstration States Actions Transitions Rewards • Directing agents to • Balance- • Continuous state • Time between space valid cells: incidents o minimizing response • Discrete states of times o Response: pending • Incident Service interest: incident locations o minimizing distance time o Incident occurrence traveled o Rebalancing: depots • Computation time o Responder • Travel time availability o Rebalancing triggered Institute for Software Integrated Systems 8 World-class, interdisciplinary research with global impact.

Problem Definition Given: System state, predicted spatial-temporal incident distribution Return: Action recommendation set that maximizes expected reward Reward can be fine-tuned Institute for Software Integrated Systems 9 World-class, interdisciplinary research with global impact.

Approaches to Solving SMDP Policy iteration New Work Queueing Theory (Dispatch) [1] Greedy • Will converge to best dispatch policy eventually Heuristic • Slow – must estimate state Search SimTrans [1] transition probabilities MCTS Multi- Agent A 2 (Dispatch) Monte Carlo A 1 Tree Search • Anytime algorithm • Not scalable to A 3 dynamic balancing [1] Ayan Mukhopadhyay, Zilin Wang, and Yevgeniy Vorobeychik. 2018. A Decision Theoretic Framework for Emergency Responder Dispatch. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS ’18). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 588–596. Institute for Software Integrated Systems 10 World-class, interdisciplinary research with global impact.

Approach 1: Greedy Search with Queue-Heuristic Institute for Software Integrated Systems 11 World-class, interdisciplinary research with global impact.

Approach 1: Queue Theory Heuristic Search Intuition μ Waiting Area Server Ambulances ‘serve’ Incidents Placed in Grid Cell incident rate := υ incidents with mean Waiting Queue rate := μ Institute for Software Integrated Systems 12 World-class, interdisciplinary research with global impact.

Approach 1: Queue Theory Heuristic Search Intuition μ Server M/M/c Queue Multiple Servers Formulation Institute for Software Integrated Systems 13 World-class, interdisciplinary research with global impact.

Approach 1: Queue Theory Heuristic Search Intuition Queue Response time: Avg time in system μ Server M/M/c Queue Multiple Servers Formulation Institute for Software Integrated Systems 14 World-class, interdisciplinary research with global impact.

Approach 1: Queue Theory Heuristic Search “Multi Class, μ 1 Multi Server Queue Formulation” μ 2 μ k Institute for Software Integrated Systems 15 World-class, interdisciplinary research with global impact.

Approach 1: Queue Theory Heuristic Search μ 1 υ 22 How to determine μ 2 υ 1 split of rates? 2 υ 3 2 υ 2 μ k Institute for Software Integrated Systems 16 World-class, interdisciplinary research with global impact.

Approach 1: Queue Theory Heuristic Search • For each cell, distribute rate among depots inversely proportional to the distance from the cell to the depot • Closer depots => higher portion of rate • Solve System of Linear equations above for each cell • υ gd is the fraction of arrival rate for cell g that is shared by depot d Institute for Software Integrated Systems 17 World-class, interdisciplinary research with global impact.

Approach 1: Queue Theory Heuristic Search • To score a particular allocation of agents: • Must consider travel times => not memoryless, so model explicitly • ϒ represents collection of split rates • Score π ϒ => sum across all cells and depots • Estimated (queue) response times (waiting + service time) • Travel time from depot to cell Institute for Software Integrated Systems 18 World-class, interdisciplinary research with global impact.

Approach 1: Queue Theory Heuristic Search • Depot selection: Greedy Search • One by one select depot that minimizes π ϒ • Add to chosen set • Re-split rates and calculate new scores with each new depot placed • Continue until the number of depots chosen is the same as number of agents • Assign agents to chosen depots by minimizing distance traveled (Linear Program) Institute for Software Integrated Systems 19 World-class, interdisciplinary research with global impact.

Approach 1: Overview Choose depots via greedy search Assign agents to chosen depots Choose depots via greedy search Assign agents to chosen depots • Repeat until # chosen depots == # • Minimize distance traveled agents: • LP, Greedy Search, etc. • Split incident rates across depots • Score allocations • Add depot that minimizes score Institute for Software Integrated Systems 20 World-class, interdisciplinary research with global impact.

Approach 1: Overview Choose depots via greedy search Assign agents to chosen depots Choose depots via greedy search Assign agents to chosen depots Advantage: • Repeat until # chosen depots == # • Minimize distance traveled agents: • LP, Greedy Search, etc. • Computationally efficient • Split incident rates across depots • Score allocations Disadvantages: • Doesn’t take internal system state into account • Add depot that minimizes score • Ignores dynamic incident rate distribution Institute for Software Integrated Systems 21 World-class, interdisciplinary research with global impact.

Approach 2: Multi-Agent Monte Carlo Tree Search (MMCTS) Institute for Software Integrated Systems 22 World-class, interdisciplinary research with global impact.