APPLYING BIG DATA ANALYTICS (BDA) TO DIAGNOSE HYDRO- METEOROLOGICAL - PowerPoint PPT Presentation

APPLYING BIG DATA ANALYTICS (BDA) TO DIAGNOSE HYDRO- METEOROLOGICAL RELATED RISK DUE TO CLIMATE CHANGE MOHD ZAKI M AMIN National Hydraulic Research Institute of Malaysia Ministry of Natural Resources & Environment OCT . 19, 2016

OVERVIEW OF CLIMATE OVERVIEW OF CLIMATE RELATED 1 1 RELATED DISASTER DISASTER SETTING THE SCENE – CLIMATE SETTING THE SCENE – CLIMATE 2 2 CHANGE AND BDA CHANGE AND BDA BIG DATA ANALYTICS (BDA) – BIG DATA ANALYTICS (BDA) – 3 3 PROOF OF CONCEPT PROOF OF CONCEPT POTENTIAL IMPACT OF CC – POTENTIAL IMPACT OF CC – 4 4 BDA FINDING BDA FINDING 5 5 WAY FORWARD WAY FORWARD

Worldwide Natural Catastrophes 1980 – 2014 Source: Munich Re

Loss Event Worldwide 2014 Geographical overview Floods East Coast of PM Dec 14-31, 2014

Kelantan-Pahang Floods, Malaysia Dec 14-24, 2014.. Continuous heavy downpour & upstream flooding.. > many properties & infrastructures Source: SREX Report (IPCC, 2011) destroyed.. 25 deaths..

Natural Disaster in Malaysia Facts: Wildfire 4 1. 76 disaster recorded in the period of 1965-2016 2. Type of disaster - wildfire, storm, landslide, Storm 3 mudflows, floods, epidemic, tsunami & drought 3. More than half of the disaster were floods hazard Mass movement (wet) 4 (45) Mass movement (dry) 5 Flood 45 Epidemic 11 Earthquake (seismic… 2 Drought 2 Flood (Dec 1965) 3,00,000 Flood (Dec 1970) 2,43,000 Flood (Dec 2014) 2,30,000 Flood (Jan 1967) 1,40,000 Flood (Jan 2007) 1,37,533 Flood (Dec 2006) 1,00,000 Flood (Nov 1988) 60,000 Flood (Nov 2005) 30,000 Flood (Dec 2007) 29,000 Flood (Nov 1986) 25,000 0 1,00,000 2,00,000 3,00,000 4,00,000

Priorities for Action Focused action within and across sectors by States at local, national, regional and global levels Priority Action 1 Priority Action 2 Priority Action 3 Priority Action 4 Enhancing disaster preparedness Understanding disaster Strengthening disaster risk Investing in disaster risk for effective response, and to risk reduction for resilience reduction for resilience “Build Back Better” in recovery, rehabilitation and reconstruction Roles of Stakeholders Civil society, volunteers, organized voluntary work Academia, scientific Business, professional Media to take a role in organizations and community-based organizations to and research associations and private contributing to the public participate (In particular, women, children and youth, entities and sector financial institutions to awareness raising persons with disabilities, and older persons) networks to collaborate collaborate Global Targets • Seven concrete global targets were specified ① The number of deaths • The targets include important policy focuses, such as ② The number of affected people mainstreaming DRR , prior investment, “Build Back Better”, ③ Economic loss multi- stakeholders’ involvement, people-centered approach, ④ Damage to medical and educational and women’s leadership facilities ⑤ National and local strategies ⑥ Support to developing countries ⑦ Access to early warning information disaster resilient community. Five colors indicate the five priority actions of the “Hyogo Framework for Action” (HFA).

OVERVIEW OF CLIMATE RELATED 1 DISASTER SETTING THE SCENE – 2 CLIMATE CHANGE AND BDA BIG DATA ANALYTICS (BDA) – 3 PROOF OF CONCEPT POTENTIAL IMPACT OF CC – 4 BDA FINDING 5 WAY FORWARD

Big Data Analytics – Initiative by the Government 23 April 2015 25 January 2015 MAMPU, MDEC and MAMPU-MDEC- 19 November 2014 MIMOS signed a MIMOS launched the MOU implement a BDA-DGL. Four (4) strategic government agencies Flagship Application participating in collaborative work Coordination Proof-of-Concept through BDA-Digital Committee (FCC) BDA initiatives were Government Open 14 November 2013 Meeting agreed of recognized Innovation Network the need to develop (BDA-DGOIN) expertise and BDA The Prime Minister has announced Centre of Excellence the Big Data Analytics initiatives in Malaysia while chairing the 25 th MSC Malaysia Implementation Council Meeting (ICM) to address the current challenges through the use of BDA technology. MAMPU has been appointed as BDA project leader for the Public Sector. MAMPU – MALAYSIAN ADMINISTRATIVE MODERNIZATION AND MANAGEMENT PLANNING UNIT MDEC – MALAYSIA DIGITAL ECONOMY CORPORATION MIMOS – GOVERMENT OWNED COMPANY (GOC)

Project Objective To develop a BDA related system that will be able to assist NAHRIM in visualizing and analyzing almost 1450 simulation-years of grid-based projected hydro-climate data for Peninsular Malaysia

Studies & Reports  Climate change impact on the hydrologic regime and water resources for Peninsular Malaysia (NAHRIM, 2006)  Climate change impact on the hydrologic regimes, water resources and landuse for Sabah & Sarawak (NAHRIM, 2010)  Study of the impact of climate change on sea level rise at Peninsular Malaysia and Sabah & Sarawak (NAHRIM, 2010)  Climate change impact on the hydrologic regime and water resources for Peninsular Malaysia (NAHRIM, 2014)

Main Data Output ECHAM5 Temperature HPC SYSTEM AGRI/PUBLIC HEALTH Soil Water Storage AGRI/FORESTRY Rainfall /BIODIVERSITY Evapotranspiration Runoff W-RESOURCES & Flow /INFRA/ENERGY ENERGY- HYDROPOWER  covers 3,888 grids @ 6x6 km area & basin scale  5 main data/parameters  Historical and future period of 1970-2000 & 2010-2100 13  1450 simulation – year in hourly increments

….precipitation change…. 10-yr Avg. 10-yr Avg. 10-yr Avg. (1980 – 1990) (1970 – 1980) (1990 – 2000)

….air temperature change…. 10-yr Avg. 10-yr Avg. 10-yr Avg. (1970 – 1980) (1990 – 2000) (1980 – 1990)

OVERVIEW OF CLIMATE RELATED 1 DISASTER SETTING THE SCENE – CLIMATE 2 CHANGE AND BDA BIG DATA ANALYTICS (BDA) 3 – PROOF OF CONCEPT POTENTIAL IMPACT OF CC – 4 BDA FINDING 5 WAY FORWARD

Big data 4V’s OF BIG DATA analytics is the process of examining large data sets to uncover hidden patterns, unknown correlations , market trends, RUN customer RAIN FLOW OFF preferences and other useful business PROJECTED HYDRO-CLIMATE DATA information (source: whalts.com)

BUSSINESS CASE IDENTIFY VISUALISE Flood flow 11 3,888 grids for river basins Peninsular and 12 states Malaysia (6x6 in Peninsular km area) Malaysia TRACE DETECT Drought Extreme episode from rainfall and weekly to runoff annual rainfall projection data for 90 data for 90 years years Visual Analysis for 190 million records

DATA USED (VOLUME) TOTAL RECORDS (OF 3 PARAMETERS) 14 Other SRES Scenarios ECHAM5 A1B POC for BDA [VALUE], [PERCENTAGE] POC for BDA 3,000,000,000 [VALUE], Parameters: • [PERCENTAGE] Rainfall • Runoff • Streamflow

POC Data Warehouse Infrastructure Data Data Cleansing & Data Repository Analytics Presentation Acquisition Integration Mi-Galactica CC Projected data (Rainfall, Runoff, Mi-Galactica Streamflow & etc.) Extraction Process Users GPU Parallel Columnar Data Staging Store Data Scientists Data PostgreSQL User Transform Administrators Meta Authentication and Load Data Data Multi-Core Many-Core Secured by a centralized authentication platform, Mi-UAP CPU GPU Volume of Powered by accelerated heterogeneous computing platform, Mi-Galactica Data

System Overview Step 4. NAHRIM access the system through Internet • Drought • Rainfall Storm Center • Rainfall and Runoff Step 3. Host • River flow analysis to Web Server Rainfall, Runoff & Streamflow Dataset Step 2. Data Processing & Acceleration using Step 1. Load NAHRIM dataset MIMOS platform (Mi- to MIMOS Platform Galactica)

OVERVIEW OF CLIMATE RELATED 1 DISASTER SETTING THE SCENE – CLIMATE 2 CHANGE AND BDA BIG DATA ANALYTICS (BDA) – 3 PROOF OF CONCEPT POTENTIAL IMPACT OF CC 4 – BDA FINDING 5 WAY FORWARD

PROJECTED HYDROCLIMATE DATA ANALYSIS & VISUALISATION FOR POTENTIAL DROUGHT & FLOOD EVENTS IN PENINSULAR MALAYSIA

FLOOD EVENT DEC 2014 – STORM PATTERN BASIN DAILY RAINFALL HISTOGRAM - SG KELANTAN (14 - 24 DEC 2014) 199.5 163.4 156.9 157 140.5 102.1 67.8 62.5 55.6 14.7 7.6 14 15 16 17 18 19 20 21 22 23 24 December 2014 22 DEC 2014 23 DEC 2014

Rainfall threshold: Average Rainfall threshold: Average Rainfall threshold: Average Threshold value: 160mm Threshold value: 140mm Threshold value: 120mm Year: 2028 Year: 2035 Year: 2031

20 Dec 2031, 168.2mm 24 Dec 2031, 160.4mm

2016 2024 Jan-Mar Apr-Jun Jul-Sep Oct-Dec Jan-Mar Jul-Sep Apr-Jun Oct-Dec 2010-2100

RAINFALL RUNOFF 18 Oct 2031 20 Oct 2031

OVERVIEW OF CLIMATE RELATED 1 DISASTER SETTING THE SCENE – CLIMATE 2 CHANGE AND BDA BIG DATA ANALYTICS (BDA) – 3 PROOF OF CONCEPT POTENTIAL IMPACT OF CC – 4 BDA FINDING WAY FORWARD 5

Way Forward VISUALISE - BDA - POC IDENTIFY Climate Change Factor (CCF) DEGREE 0F VULNERABILITY Water Stress Water - DSS DASHBOARD OF ADAPTATION SIMULATION Benefits  Sharing of data to harness the vast potential data  Sharing information makes decision making more efficient  Improved decision making process through data linkahes, data mining, data analytics and predictive analytics  Decision making is more proactive and timely manner