OGCIO Seminar on PSI/Data.One PSI and Big Data Jiannong Cao Department of Computing Hong Kong Polytechnic University
Big Data TB ( 10 12 bytes) → PB ( 10 15 bytes) → EB ( 10 18 bytes) → ZB ( 10 21 bytes) ... The World creates about 2.5EB data every day The total amount of global data reached 2.8ZB — report by International Data Corporation(IDC) in Dec. 2012 Where does the data come from Internet & Web Sensing & IoT (M2M) Enterprises Government Slide 2
Opportunities of Big Data Answer formerly unanswerable questions Traditional applications but with real-time processing of big volume of data, faster and more accurate Formulate new questions New applications where solutions exist only when volume of data becomes big enough Discover invisible knowledge New knowledge is discovered from different separate sources of data Make evidence based predictions / decisions In particular, shift from causality (why) to correlations (what happens) Slide 3
Big Data Difficulties in big data research Acquire Cannot get the data Decide Lack of effective means for Organize analyzing the data Analyze Limited use of the data PSI provides open data sets for big data research Slide 4
PSI Initiatives Hong Kong Beijing Shanghai Slide 5
Hong Kong PSI Dataset 15 Categories Charitable fund-raising Air pollution indices Buildings Geo-referenced Food and environmental activities public facility data hygiene Image resources Law and order Marine News and Population Census information statistics Property market Public Real-time Water Weather statistics transport traffic data quality data Slide 6
Beijing PSI Dataset 14 categories Tourism Dining Transportation Healthcare Entertainment Shopping Religion Education Social Welfare Living Services Rural Residents Housing Environment Enterprise Slide 7
Shanghai PSI Dataset 7 categories Transportation Healthcare Law and Order Enterprise Environment Statistics Property Slide 8
PSI Datasets PSI Data Format Data Access Development Support Beijing files Direct download API for map, (e.g. csv, XML) transport, search Shanghai files Direct download N/A (e.g. csv, XML) Hong Kong files Direct download N/A (e.g. csv, XML, mdb) RSS subscription Slide 9
Opportunities for Big Data Research Tourism Business Strategy Green City Education Easy Life Healthcare Slide 10
Tourism • Public Toilet • Quality Dining • Shopping • Cheapest Route • Free Wi-Fi Geo-referenced Food and Environmental Shopping Public Public Facility Data Hygiene Malls Transport Slide 11
Green City • Traffic Scheduling • Anti-Pollution Action • Navigation Air Pollution Indices Real-time Traffic Data Slide 12
Business Strategy • Retail Store Planning • Investment • Market Research Buildings Property Market Food and Environmental Population Census Statistics Hygiene Statistics Slide 13
Easy Life • Parking • Route Plan • Leisure Air Pollution Weather Real-time Traffic Geo-Referenced Public Data Data Facility Data Indices Slide 14
PSI-enabled Big Data Research Big data analytics for Joint analysis on urban urban transportation air quality & traffic Real-time Traffic Data Air Pollution Indices Data Slide 15
PSI-enabled Big Data Research Big data analytics for urban transportation Urban transportation planing & design With PSI on real-time traffic data, we can mine the correlation between two roads detect the congestion points evaluate the effectiveness of urban traffic planning Intelligent transportation system With PSI on real-time traffic data, we can analyze congestion state of traffic flows provide a user with the practically fastest route to a destination at a given time Slide 16
Traffic State in Hong Kong Slide 17
PSI-enabled Big Data Research Urban air quality estimation With the PSI on real-time traffic and air pollution data, we can learn the historical and real-time air quality data estimate the fine-grained air quality information based on limited number of monitoring stations predict air quality information throughout the city in future Joint analysis on urban air quality & traffic With the PSI on real-time traffic and air pollution data, we can find out the relationship between urban air quality & traffic estimate the fine-grained air quality with traffic information calculate maximal traffic volume with air quality constraints Slide 18
Air quality monitoring stations in HK — from Environmental Protection Department of the Government of the HKSAR Slide 19
Air quality statistics of areas in HK Roadside API of Mong Kok in HK Roadside API of Central in HK 120 120 100 100 80 80 60 60 2012 2012 40 2011 40 2011 20 20 0 0 Months Months Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec General API of Sha Tin in HK General API of Yuen Long in HK 70 70 60 60 50 50 40 40 2012 2012 30 30 2011 2011 20 20 10 10 0 0 Months Months Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Slide 20
Example of joint analysis on air quality & traffic Roadside API of Mong Kok in HK Roadside API of Central in HK 120 120 100 100 80 80 60 60 2012 2012 40 2011 40 2011 20 20 0 0 Months Months Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec General API of Sha Tin in HK General API of Yuen Long in HK 70 80 60 50 2012 2012 40 30 2011 2011 20 10 0 Months Months Jan Mar May Jul Sep Nov -10 Jan Mar May Jul Sep Nov Slide 21
Conclusion PSI sheds new light on big data research Various data sources Valuable data PSI offers a lot of opportunities Operational efficiency and productivity Public utilities to reduce consumption Produce safety from farm to fork Value for money for citizens Customize actions based on population segments Fraud detection and prevention Prevent crime waves Slide 22
Slide 23
Recommend
More recommend
