Comparing Cohorts of Event Sequences A VISUAL ANALYTICS APPROACH - PowerPoint PPT Presentation

Comparing Cohorts of Event Sequences A VISUAL ANALYTICS APPROACH presented by Sana Malik with Fan Du, Catherine Plaisant, and Ben Shneiderman May 26, 2016 — HCIL 33 rd Annual Symposium, College Park

often, analysts compare cohorts within datasets

any groups of users, patients, or records often, analysts compare cohorts within datasets

?

FREQUENT PATTERNS

ABSENCE OF EVENTS

DURATION

Data Collection Statistics Cohort Selection

Data Collection Visual Analytics Statistics Cohort Selection

Data Collection Visual Analytics Statistics Cohort Selection

EVENTFLOW Monroe et al. Temporal event sequence simpli fi cation . IEEE Transactions on Visualization and Computer Graphics (TVCG 2013).

EVENTFLOW Monroe et al. Temporal event sequence simpli fi cation . IEEE Transactions on Visualization and Computer Graphics (TVCG 2013).

EVENTFLOW ? Monroe et al. Temporal event sequence simpli fi cation . IEEE Transactions on Visualization and Computer Graphics (TVCG 2013).

Data Collection Visual Analytics Statistics Cohort Selection

Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Exit 38.37, 0.0, 4.11e-123 Emergency -> ICU -> Exit 24.61, 0.0, 2.11e-73 Emergency -> Normal Floor Bed -> Exit -> ICU 5.26, 0.0, 4.12e-15 Emergency -> Normal Floor Bed -> ICU -> Exit 5.26, 0.0, 4.12e-15 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> Exit 7.89, 3.22, 1.20e-06 Aspirin -> Emergency -> ICU -> Intermediate Care -> Exit 4.15, 0.0, 4.22e-12 Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 2.97, 0.0, 7.02e-09 Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.01, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 0.76, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> Exit 0.0, 1.61, 3.37e-05 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 2.37, 2.84e-07 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 3.82, 6.45, 0.00 Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 4.07, 7.17e-12 Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 4.24, 2.48e-12 Emergency -> Exit 0.0, 11.88, 9.00e-34 Emergency -> ICU -> Normal Floor Bed -> Exit 0.0, 16.12, 2.18e-46 Aspirin -> Emergency -> Exit 0.0, 47.79, 2.49e-162 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 0.59, 0.02 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU ->

5.26, 0.0, 4.12e-15 Emergency -> Normal Floor Bed -> ICU -> Exit 5.26, 0.0, 4.12e-15 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> Exit 7.89, 3.22, 1.20e-06 Aspirin -> Emergency -> ICU -> Intermediate Care -> Exit 4.15, 0.0, 4.22e-12 Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 2.97, 0.0, 7.02e-09 Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.01, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 0.76, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> Exit 0.0, 1.61, 3.37e-05 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 2.37, 2.84e-07 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 3.82, 6.45, 0.00 Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 4.07, 7.17e-12 Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 4.24, 2.48e-12 Emergency -> Exit 0.0, 11.88, 9.00e-34 Emergency -> ICU -> Normal Floor Bed -> Exit 0.0, 16.12, 2.18e-46 Aspirin -> Emergency -> Exit 0.0, 47.79, 2.49e-162 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 0.59, 0.02 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.01, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 0.76, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> Exit 0.0, 1.61, 3.37e-05

SAS STATA SAS Business Analytics Software. Vers. 9.4. SAS StataCorp. 2015. Stata Statistical Software: Institute, 2014. Computer software. Release 14. College Station, TX: StataCorp LP.

Data Collection Visual Analytics Statistics Cohort Selection

Visual Analytics Data Collection Cohort Selection Statistics

HIGH-VOLUME Hypothesis Testing

HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS

HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS REAL-WORLD Case Study

HIGH-VOLUME Hypothesis Testing

Emergency Room

Normal Floor Bed

ICU

Discharged

start and end of record

non-consecutive (contains other events between)

1 SHORT SEQUENCE 14 UNIQUE PATTERNS non-consecutive (contains other events between)

RECORD COVERAGE Does this sequence occur in more records in one cohort than the other? DURATION On average, does this sequence take longer in one cohort than the other? FREQUENCY On average, does this sequence occur more frequently per record in one cohort than the other?

RECORD COVERAGE Does this sequence occur in more records 14 UNIQUE PATTERNS in one cohort than the other? DURATION X 3 METRICS On average, does this sequence take longer in one cohort than the other? 42 HYPOTHESES FREQUENCY On average, does this sequence occur more frequently per user in one cohort than the other?

HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS REAL-WORLD Case Study

Systematic Exploration OF RESULTS

Demo

HIGH-VOLUME Hypothesis Testing Systematic Exploration OF RESULTS REAL-WORLD Case Study

REAL-WORLD Case Study

MULTI-DIMENSIONAL IN-DEPTH LONG-TERM CASE STUDIES (MILCS) Entry Interview & Training (1 session) Partners Use Tool Partners Provide Feedback (3 months) Researchers Re fi ne Tool Exit Interview (1 session) For Partners For Researchers Demonstrate utility, re fi ne tool Papers, insights, discoveries B. Shneiderman and C. Plaisant. Strategies for evaluating information visualization tools: Multi- dimensional in-depth long-term case studies. In BELIV ’06: Proceedings of the 2006 AVI workshop on BEyond time and errors, pages 1–7. ACM, 2006.

CASE STUDY PARTNERS

CASE STUDY PARTNERS

PARTICIPANTS & DATASET Three analysts at Adobe • One experienced user • Two novice users Users’ events on a product website viewing the display ads • signing up for promotions or free trials • purchasing products • Dataset Size • 6,999 users • 124 events types / 81,563 events

GOAL Compare users who purchased a product with using trials versus without using trials to understand ad-related behaviors

SYSTEM USE

SYSTEM USE

SYSTEM USE “Event filtering was the most helpful to focus the analysis”

SYSTEM USE “Reduced metric calculation time provided a much better user experience for data analysis”