A SIMULATION-BASED ARCHITECTURE FOR SMART CYBER- PHYSICAL SYSTEMS - PowerPoint PPT Presentation

A SIMULATION-BASED ARCHITECTURE FOR SMART CYBER- PHYSICAL SYSTEMS THOMAS GABOR, LENZ BELZNER, MARIE KIERMEIER, MICHAEL TILL BECK, ALEXANDER NEITZ LMU MUNICH INSTITUTE FOR INFORMATICS

SMART CYBER-PHYSICAL SYSTEMS E.g. autonomous factories, smart citjes, ... Partjally uncontrolled environment Incomplete design-tjme knowledge Complex decisions at runtjme Space of behavioral solutjons available Task: Choose adequate solutjon in tjme 2

CHALLENGES ● Challenge 1: Changing environment ● Non-statjonary optjmal behavior ● Challenge 2: Aleatoric uncertainty ● Optjmal behavior is probabilistjc ● Irreducible ● Challenge 3: Epistemic uncertainty ● “Small data” ● Reducible ● Challenge 4: Behavioral uncertainty ● Due to resource constraints 3

A SIMULATION-BASED ARCHITECTURE FOR SMART CYBER- PHYSICAL SYSTEMS

MOTIVATION Planning ● Given an MDP ● State space S, actjon space A ● Model of domain dynamics P(S|S x A) ● Reward functjon R : S x A x S → (encoding system goals) ℝ ● Find strategy P(A|S) that maximizes expectatjon of gathered reward Challenge ● Partjally uncontrolled domains ● Environment changes without system taking actjon ● Necessitates effjcient decision making 5

ONLINE PLANNING Simulatjon-Based Online Planning ● Interleaving of planning and executjon Executjon Planning ● Concentrates planning efgort on current situatjon ● Fast planning by sampling the search space State of the Art (excerpt) ● Discrete Bandid-Based Planners: MCTS, UCT, SHOT ● Rolling Horizon Evolutjonary Algorithms ● Contjnuous Bandid-Based Planners: HOOT, HOLOP ● Cross Entropy Open Loop Planning Kocsis, Levente, and Csaba Szepesvári. "Bandit based monte-carlo planning." Machine Learning: ECML 2006. Chaslot, Guillaume. "Monte-carlo tree search." Diss. Universiteit Maastricht. 2010. Cazenave, Tristan. "Sequentjal halving applied to trees." Computatjonal Intelligence and AI in Games. 2015. Perez Liebana, Diego, et al. "Open Loop Search for General Video Game Playing." ACM GECCO. 2015. Mansley, Christopher R., Ari Weinstein, and Michael L. Lituman. "Sample-Based Planning for Contjnuous Actjon Markov Decision Processes." ICAPS. 2011. Weinstein, Ari, and Michael L. Lituman. "Bandit-Based Planning and Learning in Contjnuous-Actjon Markov Decision Processes." ICAPS. 2012. Weinstein, Ari, and Michael L. Lituman. "Open-loop planning in large-scale stochastjc domains." AAAI. 2013. 6

THE DIGITAL TWIN ARCHITECTURE 7

THE DIGITAL TWIN ARCHITECTURE 8

CONTROL TIERS ● Four tjers of control ● Physical necessity ● Laws of nature, domain inherent ● In partjcular important for high-fjdelity simulatjons ● Machine-environment interface ● Circuit control, sensors, actuators, computatjonal hardware ● Is a system design variable (in contrast to physical necessity) ● Fixed in hardware, not easily changed at runtjme ● Immediate reactjon ● Watchdogs, fjxed behavioral rulesets, expert systems ● Sofuware, changeable at runtjme ● Planned reactjon ● Utjlity functjons, online planning, use of the digital twin ● Sofuware, changeable at runtjme 9

CHALLENGES, APPROACHES & CASE STUDIES

CHALLENGES & CASE STUDIES Challenges for digital twin control ● Scalability to complex models → adaptjve abstractjon ● Safe operatjon → risk awareness Case Studies ● Smart factory agent control ● Smart grid energy commitment 11

TIME ADAPTIVE MOTION PLANNING ● Consider a smart factory ● Materials and machines move probabilistjcally ● Model based motjon planning ● Problem: ● Environment changes contjnuously ● Optjmality of behavior changes contjnuously ● Simulatjon is expensive ● Goal: Collect targets ● Constraint: Avoid statjc and dynamic obstacles ● How to actuate in order to collect targets and avoid obstacles? ● We answer this questjon using a simulatjon @ runtjme ● Evaluate potentjal actuatjons with the digital twin ● Use of tjme adaptatjon for situatjve abstractjon 12

TIME ADAPTIVE MOTION PLANNING ● Fixed temporal resolutjon is not always optjmal ● Typical hyperparameter problem: Where to fjx? ● Assumptjon: A simulatjon is available as a functjon of tjme P(S|S x A x T) ● Idea: Incrementally identjfy 'important' moments ● Optjmize actjon duratjon ● Yields adaptatjon of simulatjon efgort to current situatjon ● A similar argument can be made for simulatjon depth Belzner, Lenz. “Time-Adaptjve Cross Entropy Planning.” ACM SAC 2016. 13

RESULTS With/without temporal abstractjon for empirical worst/best case parameters → Temporal abstractjon reduces tjme needed to collect the items 14

RESULTS Temporal abstractjon reduces simulatjon efgort → Enables efgectjve use of simulatjon 15

RISK-AWARE COMMITMENT IN THE GRID ● Consider a smart grid ● Highly volatjle/variable energy consumptjon ● Contjnuous reconfjguratjon of energy productjon ● Problem: ● Overproductjon induces cost ● Power supply has to be guaranteed ● Only a limited quantjty can be sold to other markets ● Goal: Minimize productjon – consumptjon ● Constraint: 0 < productjon – consumptjon < maximum oversupply ● How much power to produce on next commitment? ● We answer this questjon using a simulatjon @ runtjme ● Evaluate potentjal productjon allotments with the digital twin ● We compared risk-neutral and risk-aware strategies ● Both use simulatjons, but difger in their evaluatjon 16

RISK-AWARE COMMITMENT IN THE GRID 17

ABSTRACTION, SAFETY & UNCERTAINTY ? Safe adaptjve abstractjon ● Adaptjve abstractjon may provide scalability ● Potentjally “abstractjng away” risks? ● How to provide (statjstjcal) guarantees? Learning/adaptjng models @runtjme ● Machine learning enables model adaptatjon @runtjme ● However, this introduces model uncertainty ● How to treat safety issues under model uncertainty? ● E.g. behavior optjmizatjon vs. model confjdence ● Possible approach → QoS-awareness ● Optjmize “up to” QoS requirement ● Maximize confjdence ● Consequences of adaptjve abstractjon and model uncertainty? 18

SUMMARY & NEXT Summary ● Smart simulatjon-based CPS ● Digital twin architecture ● Control tjers ● Applicatjon examples ● Challenges for simulatjon-based control Up next ● QoS-awareness, V&V @runtjme ● Learning models @runtjme ● Combining abstractjon & safety ● Distributed simulatjon-based CPS 19