Platforms for Data Sharing, Data Analytics, and Data Visualization - PowerPoint PPT Presentation

Platforms for Data Sharing, Data Analytics, and Data Visualization Richard Glassco Noblis Consultant to FHWA Office of Operations: Research & Development

Agenda  Overview of the Safety Pilot Model Deployment  Size-Related Considerations  Demonstration of the RDE  Other CDS Program Tool development activities U.S. Department of Transportation 2 ITS Joint Program Office

R ESEARCH D ATA E XCHANGE U.S. Department of Transportation 3 ITS Joint Program Office

Goals of the Safety Pilot Model Deployment 1. Test connected vehicle operations in real-world conditions 2. Understand how regular drivers use connected vehicle technologies 3. Determine the safety benefits of a connected vehicle U.S. Department of Transportation 4 ITS Joint Program Office

Ann Arbor, Michigan, site of SPMD U.S. Department of Transportation 5 ITS Joint Program Office

SPMD T YPES OF M ESSAGES  Basic Safety Messages (BSMs)  Signal Phase and Timing (SPaT) Messages  MAP messages  Traveler Information Messages (TIMs) Plus contextual data:  Weather  Traffic counts Plus documentation U.S. Department of Transportation 6 ITS Joint Program Office

SPMD Statistics • 2836 vehicles driving on 73 miles of roadways (freeways and arterials) • Each vehicle generates Basic  The SPMD data environment is Safety Messages (position, structured in 6 data sets, with a speed, acceleration, etc.) every total sanitized volume of 10 th of a second approximately 24 GB • 2 Months of data (4 billion  The original un-sanitized data BSMs) are on the RDE set was approximately 50GB U.S. Department of Transportation 7 ITS Joint Program Office

SPMD Data in the RDE U.S. Department of Transportation 8 ITS Joint Program Office

SPMD Contained Data that Could be Shared •Some Data may not be shared, for example: •It contains PII that cannot be removed •It contains commercially sensitive information •It contains intellectual property rights •There are legal reasons the data cannot be released to the public •Such data may be stored in the FHWA’s Saxton Transportation Operations Laboratory. Access to data in the Saxton Lab is restricted to researchers with appropriate qualifications and training. U.S. Department of Transportation 9 ITS Joint Program Office

Requirements for Data Submitted to the RDE  PII is data that by itself or in combination with other data could be used to determine the identity of individual private vehicles or persons from who driving data is obtained.  Examples of PII include driver age, gender, social security number, or address, and vehicle VIN number, license plate number, exact length and width, or weight, or other identifying characteristics.  PII may also be derived from vehicle trajectories that reveal the origin or destination of personal trips, or intermediate stops of personal nature such as a school, workplace, or frequent errand.  Public transit vehicles and public vehicles such as maintenance trucks are not subject to this PII restriction U.S. Department of Transportation 10 ITS Joint Program Office

PII had to be removed from the SPMD data while PII had to be removed from the SPMD data while PII had to be removed from the SPMD data while maintaining meaningfulness of the data maintaining meaningfulness of the data maintaining meaningfulness of the data  To protect participants’ identity the RDE  To protect participants’ identity the RDE team rid all data files of data elements team rid all data files of data elements that contain PII that contain PII  Data elements that could be paired with  Data elements that could be paired with other publicly available data were also other publicly available data were also deleted deleted  Vehicle trajectories, with points collected  Vehicle trajectories, with points collected at 10Hz, revealed the identity of at 10Hz, revealed the identity of participants, therefore participants, therefore □ Sanitization algorithms were □ Sanitization algorithms were developed to truncate trajectories to developed to truncate trajectories to mask trip origins and destinations mask trip origins and destinations □ The algorithms were also applied to □ The algorithms were also applied to dependent / related data elements dependent / related data elements U.S. Department of Transportation 11 ITS Joint Program Office

Because of the Size of the Data Environment …  Moved to storage on the Cloud rather than contractor's server  Developed subsetting tool U.S. Department of Transportation 12 ITS Joint Program Office

Full Year of SPMD Data  USDOT has reserved the entire year of data for future study  Unsanitized, so it is not posted on the RDE – it will be stored in the Saxton Lab  Restricted access  Special restrictions for video or audio data U.S. Department of Transportation 13 ITS Joint Program Office

LIVE DEMONSTRATION OF RDE U.S. Department of Transportation 14 ITS Joint Program Office

OTHER CDS TOOL DEVELOPMENT ACTIVITIES U.S. Department of Transportation 15 ITS Joint Program Office

Data Visualization Project  Data visualization is #1 request of RDE stakeholders  Uses data on the RDE to develop data visualization tools  Map and non-map based, interactive  Applicable to multiple data environments U.S. Department of Transportation 16 ITS Joint Program Office

Samples from Data Visualization Project U.S. Department of Transportation 17 ITS Joint Program Office

Dynamic Interrogative Data Capture (DIDC) U.S. Department of Transportation 18 ITS Joint Program Office

BSM Emulator U.S. Department of Transportation 19 ITS Joint Program Office

OSADP - Companion Website for Software •www.itsforge.net U.S. Department of Transportation 20 ITS Joint Program Office

References and Contact Info •Research Data Exchange (RDE) website https://www.its-rde.net/ •Open Source Application Development Portal (OSADP): www.itsforge.net Jon Obenberger, PhD, P.E. Senior Transportation Research Engineer, FHWA Jon.Obenberger@dot.gov U.S. Department of Transportation 21 ITS Joint Program Office