Monitoring and Analyzing London's Air Quality with Pentaho Mark - PowerPoint PPT Presentation

Monitoring and Analyzing London's Air Quality with Pentaho Mark Semenenko, Pentaho Sales Engineer, Hitachi Vantara Anya Rumyantseva, Pentaho Data Scientist, Hitachi Vantara

Pentaho

Agenda Developing Pentaho Demo showcasing the platform’s end-to-end capabilities • Pentaho as an end-to-end platform • Use Case Overview • Demonstration • Summary

Pentaho provides flexible management for end-to-end data flows… Data Engineering Data Prep Analytics Data Discovery Analysis Ingestion Processing Blending Data Delivery / Analysis & Dashboards Lifecycle Data Dynamic Data Administration Security Monitoring Automation Management Provenance Pipeline

…and Machine Learning capabilities! Full automated Data Source 1 Data connectors production Data Transformation X y pred and Machine Learning Workflows Data Source 2 Dashboard decision support Data Source n Data Engineering Data Preparation Analytics

Aim: develop a demo showing a full spectrum of Pentaho capabilities

Our Use Case “London Air Quality Monitoring”

Why? • Air quality affects well-being of residents and visitors of the city • 9000 early deaths a year in London due to air pollution • Air pollution monitoring and mitigation are crucial for city councils • The use case is easy to understand and to follow

Data Sources • London Air Quality Network (96 stations across the city) • Road Traffic Counts (number and vehicle type) • Meteorological Parameters (wind, temperature, humidity)

Collecting, Preparing and Blending

Demonstration

Collecting, Preparing and Blending

Air Quality Prediction

Pentaho Machine Learning Orchestration 3 Predictive model for Air Quality was added to the demo. 1 1 2 4 *Long-Short Term Memory neural network was used to make predictions

Demonstration

Operationalising Predictive Model in Pentaho What we have shown in the demo: • Feature engineering and data preparation • Scheduled model training and update • Model application on new data streams • Tracking and storing model performance metrics • Displaying predictions in Pentaho dashboards Operationalize machine learning models in Pentaho!

Summary What we covered today: • Pentaho provides a single consistent experience for developing data products • The platform allows to put machine learning algorithms into production • Informative dashboards for decision making based on advanced analytics and data transformation in the backend