DB&A: AN OPEN SOURCE WEB SERVICE FOR METER DATA MANAGEMENT - PowerPoint PPT Presentation

DB&A: AN OPEN SOURCE WEB SERVICE FOR METER DATA MANAGEMENT Søren Aa. Mikkelsen and Rune H. Jacobsen Anders F. Terkelsen {smik,rhj}@eng.au.dk aft@haugstad-terkelsen.dk Department of Engineering Haugstad & Terkelsen ApS Aarhus University, Denmark Denmark AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

OUTLINE Introduction & scope  Data model  Common analytics for MDM  Our evaluation results  › Case study: SmartHG project › Experimental work Discussion  Conclusion & future work  AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

INTRODUCTION (1/2) Increasing number of initiatives dealing with Smart Grid  › Core: Meter data There exist many commercial solutions  › Established companies: IBM, Oracle, Microsoft › New start-ups: Energyworx, Virdata, Waylay.io No open solution for deploying value-added energy management services  › Have to “reinvent” MDM service for their purpose AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

INTRODUCTION (2/2) Our proposal  › Open Web service as a reference implementation for Meter Data Management (MDM) › Fit for use in multi-architecture › Address specific data latency scope › Considers service composability › Considers five fault scenarios › Used in an EU project called the SmartHG project for over 8 months › Evaluated in an experimental setup AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

CONTEXT: THE SMARTHG PROJECT Ecosystem of energy-aware web services Two goals: Minimise the energy usage and costs for each individual home 1. Support the Distribution System Operator (DSO) in optimising the operation of the 2. distribution grid. AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

CONSIDERED ARCHITECTURES System architectures  Cloud-based  HEMS-based  Hybrid-based AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

DATA LATENCY LEVEL (DLL) HIERARCHY Data latency = time a sensor o acquires a measurement until it is stored in the MDM DLL 0 : Firm real time – Often require closed loop controller DLL 1 : Soft real time – Degrades system’s QoS. DDL 2 and 3 : Low real time requirements Source: Courtesy of Accenture AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

RELATIONAL MODEL OF METER DATA AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

DATA MODEL FOR RESIDENTIAL HOMES Fault scenarios Meters may be attached to different 1. appliances during its life-cycle Meters can break 2. Meters are able to send data in burst- 3. mode Meters may be installed wrongly 4. Meters may sent faulty data 5. AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

THREE COMMON ANALYTICS FOR MDMS o Filtering o Condensation o Virtual Metering AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

DATA CONDENSATION Condensation – specialisation of aggregation o Common services prefer even sampled time o series Services have different granularity o requirements However, automatic data condensation will o make fault detection harder M: Measurement set Δt: Granularity t s : Start time k: Meter port t e : End time AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

VIRTUAL ENERGY PORT CONCEPT Main meter Sub meter Situation: Only when current o measurements are available at submeter level Uncertainty: Largest loads are often o inductive Aim: Higher accuracy o Constraint in calculating the virtual energy measurements: AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

IMPLEMENTATION RESTful web application  › Django 1.7.4 › Data model implemented as an app › Use of PostgreSQL database for cloud application and SQLite for HEMS › Django REST Framework › Serialisation and deserialization › User permission policies Available at › Throttling the rate of requests https://github.com/dbservice/dbservice/ › Pagination of responses AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

SMARTHG: DEPLOYMENT EXPERIENCE Deployed from February to September 2015  Experience from field trial  › Continuously roll-out of metering equipment  System must adapt › Malfunctioning  Replacement › Being shut off unintentionally or removed › Service maintenance Obtained over 80 million measurements from the test bed. AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

DATA SET CHARACTERISTICS FROM TEST BED # of measurements Sample period Percentage of data successfully retrieved End time Start time Uncertainty in estimating M Ts : Sampling period fluctuates  Meter is offline  Maintenance  Manually querying  AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

EXPERIMENTAL SETUP Cloud system Raspberry Pi Node.js client module: bench-rest AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

EXPERIMENTAL EVALUATION Subset of SmartHG data set Function Request Response Posting One measurement Replays measurement Filtering A week of consumption 20 JSON formatted data measurements and total number of measurements Condensation A week of consumption 20 JSON formatted data condensed on outputs and total daily basis number of measurements Virtual A week of virtual All entries without energy measurements pagination. AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

DISCUSSION Use of the Python language and Django framework o Impact on experimental results: o › Caching system › Reverse proxy › Placement of business logic: database queries vs. Django framework › Pagination  Minimise size of response and better distribution of computational burden › Makes evaluation comparison difficult Evaluation of MDM data latency with composability setup o AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

CONCLUSION & FUTURE WORK Complies with five faulty scenarios o Provide common analytic functions for MDM o Evaluated on a large and small scale o Source code available at: https://github.com/dbservice/dbservice o Future Work Security and privacy o › Better user model to give granular data access Quality assurance o › Use metering hierarchy to detect inconsistencies in meter data AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

QUESTIONS? smik@eng.au.dk AU AARHUS SØREN AAGAARD MIKKELSEN IEEE SOSE 2016 UNIVERSITY DB&A: AN OPEN SOURCE WEB SERVICE FOR 29 MARCH 2016 METER DATA MANAGEMENT SCIENCE AND TECHNOLOGY

AU AARHUS UNIVERSITY