Supporting decision making in security forces command center at - PowerPoint PPT Presentation

Supporting decision making in security forces’ command center at large-scale events via video-based situation monitoring and base data supply Armin Köfler 1 , Viktoria Pammer-Schindler 2 , Alexander Almer 1 , Thomas Schnabel 1 1 JOANNEUM RESEARCH Forschungsgesellschaft mbH, Steyrergasse 17, 8010 Graz, Austria 2 Graz University of Technology, Know-Center GmbH, Inffeldgasse 13, 8010 Graz, Austria Graz, 12.10.2017 1 Armin Köfler

Content • Motivation • Approach / Timeline • Prototypes / Field studies • Research Questions • Current Situation & Outlook 2 Armin Köfler

Motivation • Major events are popular and attract huge crowds • Carrying out the event in a safe manner is indispensable • All involved organizations have a need for situation information • Heads of security need information that is exact, current and unambiguous • Prevention and improved reaction on hazards through situation visualization • Project scope: KIRAS – Security Research: Project MONITOR 3 Armin Köfler

Approach (1/3) • Design Case Study • Gaining domain knowledge • Iterative process of prototype improvement and assessments by domain users • Cognitive Walkthrough • Inspection of the usability aspects of prototypes 4 Armin Köfler

Approach / Timeline (2/3) 1 2 1. Rough objective derived from project MONITOR 2. Domain-specific research work 3 3. Meeting with fire fighting experts (FF) 4. Telephone interviews with FF and Security Chief 4 5. Discussion of derived features 6. Implementation of prototype P1 after feature selection 5 6 7. Feature Demonstration (Workshop) 7 5 Armin Köfler

8 Approach / Timeline (3/3) 9 10 8. Field study “Nova Rock” 9. Field study “Donauinselfest” 10. Integration of newly recognized features in prototype P2 11 12 11. Intermediate feedback during “Frequency Festival” 12. Finalization of feature implementation 13 13. Workshop 6 Armin Köfler

Prototype P1 • A GIS as management support system • Desktop application based on C# • Focus on presenting current situation based on current data • Crowd representation derived from aerial images • Visualization and Interaction by using synthetic data • GUI Interaction oriented on widely known standard software 7 Armin Köfler

P1 – Data source ARGUS 1/2 • Prosilica GE 6600 – RGB camera (29 MPx) • INFTRATEC thermal camera (TIR) • Novatel GPS, IMAR/FSAS IMU • Line of Sight (LoS) Communication 8 Armin Köfler

P1 – Data source – ARGUS 2/2 • Example ARGUS RGB with 7cm ground resolution (600m AGL) • Aim is to extract • Positions of persons • Densities of crowds • Aim is a geographically referenced situation layer for operator 9 Armin Köfler

P1 – Crowd Visualization • Crowds are Heatmap Cells • Data basis: Raster 1x1m • Each raster cell contains • Density (#Persons / m 2 ) • Confidence of density • Site visually partitioned into • ≥ 1 P/m 2  colored region • < 1 P/m 2  transparent region • confidence < threshold  Fog of war 10 Armin Köfler

P1 – Crowd Interaction 1/2 • Heatmap Cell interpreted as unit • Features meta data 11 Armin Köfler

P1 – Crowd Interaction 2/2 • Context menu commands / functions • CMD: “Estimate cell behavior” • FUN: “Remind in 5 minutes” • FUN: “Set cell behavior manually” 12 Armin Köfler

P1 – Snipping Tool • Requested from firefighter context • Aims at efficient information distribution 13 Armin Köfler

P1 – Experts workshop – Lessons learned • Concept of heatmap cells is deemed applicable • Color schema was not favored, ZAMG schema proposed (G Y O R) • Sending security personnel to cell useful, routing support not wanted • Deferred reminding of crowd cells is endorsed • Displaying meta information is essential • The common site plan is mandatory as background layer (no OSM, …) • Snipping Tool is appreciated as additional source for event logs 14 Armin Köfler

Field Study Nova Rock • Separate containers for Security and Emergency Services • Interaction only via TETRA, radio und cell phones • Video-Streams shared between security und police Forces • Used operating resources based on paper 15 Armin Köfler

Field Study Donauinselfest • Shared container for Security and Emergency Services • Focus on verbal communication between organizations • Only ambulance service provides web interface • Used operating resources based again on paper 16 Armin Köfler

Field Studies – Lessons Learned • Master data and event log are paper based • Bringing the spatial context to mind is cumbersome during calls • (Only) verbal communication not reliable due to its volatile nature • Inspiration gained by observing specific incident: • Need for reduction of mistakes based on usage of non-volatile communication • Need for quick landmarks envisionment during radio communication 17 Armin Köfler

Prototype P2 - Features • Focus shift to presentation of master data • Efficient finding of places and reference points • Support of swift envisionment of state and location during radio call • Persistent situation information readily apparent for all of deciding staff 18 Armin Köfler

P2 – Overview Interaction Window Presentation Window (Primary Screen) (Beamer, TV) 19 Armin Köfler

P2 – Interaction Window – Map 1/2 • Map Grid defined by • 3 vertices in WGS84 • Edge length of cell in meter • Mouse position is highlighted in grid • Optional mouse label shows grid name 20 Armin Köfler

P2 – Interaction Window – Map 2/2 • (Food) booths • Extracted from official site plan of securing organizations • Labeled with booth number • Security personnel • Fixed locations based on synthetic data (mockup) • Each security person has a number visible on vest • Some have a radio at hand • Mouse position highlights objects within certain distance by an orange ring 21 Armin Köfler

P2 – Interaction Window – Find & Search Box • Canvas for mouse context information • Text search by • Name of Grid Cell • Name of all described point objects • Number of booth, Number of security person, … • Focus map on entity by hitting <Enter> • Iterate focus by repeating <Enter> on multiple eponymous objects • Red font signals no more matches possible • Example for Tool “Find nearest security with radio”: 22 Armin Köfler

P2 – Presentation Window – Overview • Presents a situation to the whole head staff as nonvolatile information • Presents situations as • Screenshot von map view including “Find & Search Box” • Timestamp of creation in tab caption • Elapsed time printed over screenshot permanently updated • Free-hand annotations per touch and mouse • Multiple situations arranged as tabs captioned by absolute time 23 Armin Köfler

P2 – Presentation Window – Situation Example 1/2 • Newly captured situation shows • Timestamp in tab title • Relevant Content of Interaction Window • Elapsed time on top 24 Armin Köfler

P2 – Presentation Window – Situation Example 2/2 • Annotated situation showing • Timestamp in tab title • Relevant Content of Interaction Window • Elapsed time on top • Annotations • Sketches 25 Armin Köfler

P2 – Expert workshop – Lessons learned • Head of security names the Presentation Window valuable • Adaption of used hotkeys advised • Search Box is rated an improvement in case processing • Highlighting master data objects on map depending on mouse distance is deemed valuable for bringing spatial context to mind • Reenacted situation with DIF-Operator allowed swifter detection of mistake • Expanding the kinds of master data is desired (Camera locations, ...) 26 Armin Köfler

Research issue 1/2 • Does the presentation of a current situation as an interactive digital map improve the situational awareness of the security operator?  Seeking reference objects speeds up spatial envisionment  Digital form is superior to paper form (but keep redundant!)  The visualization encourages reflection 27 Armin Köfler

Research issue 2/2 • How can a situation made to be known to all relevant headquarter staff? How can mistakes be mitigated / reduced?  Open presentation of situations is key  It creates a culture of obligation to fetch information for anyone  Discrepancy in presented information shall motivate to actively discuss unclear or contradictions 28 Armin Köfler

Current Situation & Outlook • Combination of the core features of prototypes into one • (Utilization of prototype at Frequency 2017) • Expert workshop by the end of October for broader feedback 29 Armin Köfler

Discussion of further development 30 Armin Köfler

Thank you for you attention 31 Armin Köfler

ANNEX 32 Armin Köfler

Control-Loop of leading 33 Armin Köfler