Crowdsourcing of Weather Data on Mobile App and Deep Learning Lior - PowerPoint PPT Presentation

Crowdsourcing of Weather Data on Mobile App and Deep Learning Lior Perez 99th AMS annual meeting

Crowdsourcing on Meteo-France mobile app Context: ■ fewer resources devoted to human observation ― Crowdsourcing can help: ■ To get a high density of human observations ― To get information on impacts of weather events ― Dedicated observation app: NO ■ Too difficult to get a large audience ― Add a crowdsourcing module in our general public app ■ Benefit from a 1M visitors per day audience ―

Keep it simple We wanted maximum participation rate ■ Keep it simple! A challenge for our culture of weather experts... In first version: Only immediate observation ■ Only for geolocalized users ■ No quantitative observation ■ Very few details in each observation ■

Feedback and gamification to increase user engagement

Success in quantity and quality 10k to 40k observations every day ■ Approx 1000 obs / h disseminated on all the French territory ■ Good quality, very few outliers ■ Large increase of observations rate in severe weather ■ (13 observations of a tornado at 1am)

Outliers filtering Methods investigated: ■ Obvious outliers removal ― For instance: Hail + Fog + Sun + Strong wind ► Anomaly detection using the multivariate gaussian ― distribution, to detect unreliable users Conclusions: ■ Most unreliable users don’t come back ― Returning users are generally reliable ― Fake observations < 1% ―

What are we doing with the data?

Internal visualization interface for forecasters

Subjective product validation Product : distinction of hydrometeors ■ No precipitation / No snow lying on the ground No precipitation / Snow lying on the ground No precipitation / Ground invisible (night or clouds) Drizzle Rain Drizzle over frozen ground Rain over frozen ground Freezing drizzle Freezing rain Rain and snow mixed Slushy snow Wet snow Dry snow Slushy snow lying on the ground Wet snow lying on the ground Dry snow lying on the ground Ice pellets Small hail Medium hail Large hail Validation of the distinction between snow and rain

New feature: Observation with picture

Enabling users to post pictures New feature: observation with picture ■

Issue We need real time moderation OK Not OK

Image classification: a problem solved by Deep Learning Dogs vs. cats ■ Cat Dog

Image classification: a problem solved by Deep Learning ImageNet: ■ Database of 14 million hand-annotated images ImageNet Challenge: ■ Image classification models, better than human performance

Transfer Learning 1) Use an image classification model that has been trained on 1.2 million of images from ImageNet Inception v3 ― 2) Re-train it: specialize it on our two classes Class 1: OK, it’s related to weather ― Class 2: Not OK, it’s not related to weather ― It’s an easy an quick process!

Prepare a training dataset: images in two folders (OK / Not OK) OK Not OK

Pictures on the app in production since November No incident, the automatic moderation system has worked ■ Accepted Rejected 0

Crowdsourcing of weather data: conclusion Excellent user feedback ■ Already used ― By forecasters ► For subjective validation ► of products Good public participation level ― Perspectives ■ Use of Deep Learning image classification to identify the type ― of weather on pictures Enable advanced users to make more detailed observations ― Lior.perez@meteo.fr