for Balance Control in Smart Grids Francisco S. Melo, Alberto - PowerPoint PPT Presentation

Decentralized Multiagent Planning for Balance Control in Smart Grids Francisco S. Melo, Alberto Sardinha, Stefan Witwicki , INEDC-ID / Instituto Superior Técnico, Porto Salvo, Portugal Laura Ramirez-Elizondo, and Matthijs Spaan Delft University of Technology, the Netherlands

Our position... Smartgrid Research Multiagent Sequential Decision-Making (MSDM) Research

Sequential Decision Making • Agents model – model their environment – receive observations observations – take actions that maximize their objective funtion actions Agent Environment (PO) MDP Model reward observation r t r t+1 o t +1 o t +2 current o t a t a t+1 state action s t s t +1 s t +2

Multiagent Sequential Decision Making model • Agents – model their environment – receive observations – take actions that maximize their objective funtion (decentralized) Environment actions and Dec-POMDP Model Agents observations team reward observations actions r t o t current state s t +1 s t transition next state

Smartgrid Agents Power Substation A Routing Actions Smart distribution Neighborhood 1 agent Weather Smart Meters Prediction Observations Observations Estimated power Weather consumption & flows Supply level production Substation B Smart distribution Power agent Routing Transmission Grid Actions Neighborhood 2 Smart Meters Energy Storage Facility

Planning with Decentralized POMDPs • Policy maps individual observations to individual action • Optimal joint policy optimizes objective function (implemented as rewards associated with states and actions) • Agents plan policies together, then execute them separately • Strengths – policies account for uncertainty in state transitions – agents need not have a perfect view of the world state – principled notion of optimality • Weakness: computational tractability

Recent Advances in MSDM • Conventional Solutions Infeasible to apply • Recent trends... – Decomposition of Models and planning processes  Distributed (policy) computation [Guestrin & Gordon 2002, Nair et al 2003, Nair et al 2005, Varakantham et al 2009, Witwicki & Durfee 2010] – Heurstic Search [Szer et al 2005, Oliehoek et al 2010, Spaan et al 2011] – Exploiting weakly-coupled interaction structure • Agent Locality [Kim et al 2006, Varakantham et al 2009, Witwicki et al 2011, Oliehoek et al 2012] • State Locality [Oliehoek et al 2008, Spaan & Melo 2008, Melo & Veloso 2011] • Degree of Influence [Becker et al 2004, Witwicki & Durfee 2010, Witwicki 2011]  Significant improvements in efficiency and scalability

Large-Scale Network of Agents

Potential Benefits for MSDM • SmartGrid network contains intrinsic structure that recent MSDM techniques should be able to exploit – Sparsely-connected compontents – Limited interaction among agents • Exciting real-world domain ripe with socially-relevant problems

Potential Benefits for Smartgrid • Decentralizing control – eliminates single point of failure – well-aligned with future smartgrid paradigms • widely-distributed power generation • diversification of energy sources • MSDM for smart control – Control policies can be planned autonomously – Accounts for uncertainty of renewable energy sources – Agents consider both the short-term and long-term impacts of their control decisions – Can optimize a balance of a variety of different objectives (e.g., economic, safety, environmental impact) – Harness the many recent MSDM algorithmic advances

Thank You • Questions? • Suggestions? Contact Info: • witwicki@inesc-id.pt (Stefan Witwicki) • jose.alberto.sardinha@ist.utl.pt (Alberto Sardinha)