Towards statistically solid thinking about validation - PowerPoint PPT Presentation

Towards statistically solid thinking about validation Rick.Donnelly@wsp.com | WSP | 6 May 2020

Traditional practice 2

Top five criteria (n=9) 3

Three revelations Rethinking validation: 1. Differentiate by count type 2. Trimming outliers 3. Using meta-networks 4

Not all counts are created equal 5

Recent outcome 6

Potential error in counts Implications for counts: Assumptions: • Data are normally distribution • Distribution has mean μ and standard deviation σ • Observations are independent of one another 7

Alternative view 8

Some traditional stats Root mean squared error: Percent root mean squared error: 9

Outlier effects on RMSE and percent RMSE 10

Optimal counts? 11

Meta-networks 1. Include only freeways, expressways, major arterials 2. Collapse freeway interchanges to single point 3. Meta-nodes: where these arcs intersect 4. Meta-links: single bidirectional arc between meta-nodes 5. One count per meta-link → independent observations by definition 6. Validate and visualize at the meta-network level 12

Forecasting errors 13

In perspective... 14

Telecommuting trends over time 15

Questions? 16