Explainable Self-Learning Self-Adaptive Systems Verena Kls | ES4CPS - PowerPoint PPT Presentation

Software and Embedded Systems Engineering Explainable Self-Learning Self-Adaptive Systems Verena Klös | ES4CPS 2019

Motivation uncertainties safety-critical autonomous dynamic networks decisions physical run-time evolution environment embedded devices TRUST ??? Dezentrales Logo optional 2 Explainable Self-Learning Self-Adaptive Systems | Verena Klös

Framework for Design and Verification of Self-Learning Self-Adaptive Systems [SEAA’15, QA4SASO’16, SEAA’17, JSA’18, JSS’18] Main idea: System • run-time evolution • comprehensibility Evolution of Adaptation Logic • formal guarantees • collected data as explanation basis Adaptation Layer Knowledge Models → Trust runtime models Managed System Environment Dezentrales Logo optional 3 Explainable Self-Learning Self-Adaptive Systems | Verena Klös

Knowledge Models • main idea: • comprehensible run-time representation of • system and environment state A P • system goals M E Knowledge Models • adaptation options • knowledge models: Managed System • K Sys , K Env : current and past system parameters & sensor values • K Goal : hierarchical quantitative goal model (distance) • K Adapt : timed adaptation rules • history: enable retracing of adaptation decicions & their context Dezentrales Logo optional 4 Explainable Self-Learning Self-Adaptive Systems | Verena Klös

Timed Adaptation Rules main idea: efficient & comprehensible model of adaptation ● options, their effect and timing timed adaptation rules as generic mechanism ● ● WHEN to apply? (guard) ● WHICH actions? (commands c i ) ● WHAT happens? (effect) ● WHEN? (time)  formally encode assumptions on environment  predictability & comprehensibility of rule-based adaptation Dezentrales Logo optional 5 Explainable Self-Learning Self-Adaptive Systems | Verena Klös

Example – Autonomous Delivery Drones ● goal model: Adaptation Goals AND w 3 w 1 w 2 Safety Delivery Battery Regulations Times Consumption AND AND AND w 13 w 11 w 12 Created by Johndory - Freepik.com . . . Flight Collision Velocity Height Avoidance AND p w 112 w 113 w 111 near airport country side city h Î [10,50] h Î [5,200] h Î [10,100] ● adaptation rule: remaining_route ≥ battery_range && reduceVelocity() → remaining_route < battery_range and estimatedDeliveryTime‘ > estimatedDeliveryTime after X seconds Dezentrales Logo optional 6 Explainable Self-Learning Self-Adaptive Systems | Verena Klös

Evolution of Adaptation Logics System Evolution of the Adaptation Layer Rule Accuracy Observation-Based Simulation-Based Verification Evaluation Learning Learning adjusted rules knowledge Adaptation Layer Analyse Plan Adaptation Goals K Env K Sys K Adapt Goal Monitor Execute provide Manager models Control Data Managed System sense control Environment Dezentrales Logo optional 7 Explainable Self-Learning Self-Adaptive Systems | Verena Klös

Explainability of Adaptation Decisions? Example ● remaining_route ≥ battery_range && reduceVelocity() → remaining_route < battery_range and estimatedDeliveryTime‘ > estimatedDeliveryTime after X seconds Questions ● 1) Why was adaptation necessary? 3) Which assumptions were made? 2) Why did you 4) Which results were achieved? choose THIS option? Dezentrales Logo optional 8 Explainable Self-Learning Self-Adaptive Systems | Verena Klös

Explainability of Adaptation Decisions? Questions ● 3) Which assumptions 1) Why was adaptation necessary? were made? 2) Why did you 4) Which results were achieved? choose this option? Knowledge needed for Answers: ● 1) cause (i.e., violated adaptation goals) 2) context, applicable rules, chosen rule, expected effect/s (in terms of observable changes & goal satisfaction) 3) chosen rule, expected effect (in terms of observable changes & goal satisfaction) 4) actual effect Dezentrales Logo optional 9 Explainable Self-Learning Self-Adaptive Systems | Verena Klös

Explainability of Learning Decisions? Questions 2) How were the ● results achieved? 1) Why was learning necessary? 4) Which results 3) Which assumptions were made? were achieved? Knowledge needed for Answers ● 1) original adaptation rule + observed deviations, non-existence of applicable rule 2) kind of learning, underlying observations or used run-time models, fitness function of the learning algorithm 3) underlying observations or used run-time models + simulation traces 4) learning result + fitness value Dezentrales Logo optional 10 Explainable Self-Learning Self-Adaptive Systems | Verena Klös

Future Work current explanations = all possibly relevant knowledge for ● each decision ( explainability = ability to explain ) – only comprehensible for experts – idea: generate textual explanations (based on language patterns) for non-experts, customize explanations for different target groups (domains, expertise with the system) useful explanations? ● Dezentrales Logo optional 11 Explainable Self-Learning Self-Adaptive Systems | Verena Klös

Useful Explanations? segmentation fault vs. Dezentrales Logo optional 12 Explainable Self-Learning Self-Adaptive Systems | Verena Klös

Future Work current explanations = all possibly relevant knowledge for ● each decision ( explainability = ability to explain ) – only comprehensible for experts – idea: generate textual explanations (based on language patterns) for non-experts, customize explanations for different target groups (domains, expertise with the system) useful explanations? ● – provide filtered information to answer questions – cooperate with cognitive science to investigate human needs and expectations on explanations – use machine learning on user feedback to learn characteristics of helpful explanations Dezentrales Logo optional 13 Explainable Self-Learning Self-Adaptive Systems | Verena Klös

References [KGG16] Klös, Verena; Göthel, Thomas; Glesner, Sabine: Formal ● Models for Analysing Dynamic Adaptation Behaviour in Real- Time Systems. In: 3rd Workshop on Quality Assurance for Self- adaptive, Self-organising Systems (QA4SASO). IEEE, pp. 106– 111, 2016. [KGG18a] V. Klös, T. Göthel, and S. Glesner, “Comprehensible ● and dependable self-learning self-adaptive systems,” Journal of Systems Architecture, vol. 85-86, pp. 28–42, 2018. [KGG18b] V. Klös, T. Göthel, and S. Glesner, “Runtime ● Management and Quantitative Evaluation of Changing System Goals in Complex Autonomous Systems,” Journal of Systems and Software, vol. 144, pp. 314–327, 2018. Dezentrales Logo optional 14 Explainable Self-Learning Self-Adaptive Systems | Verena Klös