EXTREME BEHAVIOR IN Consists of many entities that interact in - PowerPoint PPT Presentation

OVERVIEW ➤ Extreme behavior in information and communications technology ( ICT ) systems ➤ Limits of predictive risk analysis ➤ Complexity is the enemy ➤ From fragile to antifragile systems ➤ Design and operational principles ANTIFRAGILE ICT SYSTEMS ➤ Antifragile microservice systems Kjell Jørgen Hole version 1.0 2 COMPLEX ADAPTIVE SYSTEM ➤ Man-made or natural system EXTREME BEHAVIOR IN ➤ Consists of many entities that interact in involved ways ➤ Entities adapt to each other and the environment ICT SYSTEMS ➤ Adaption allows system to withstand perturbations 3 4

EXAMPLES OF COMPLEX ADAPTIVE SYSTEMS COMPLEX ADAPTIVE ICT SYSTEMS ➤ The world-wide economic system ➤ A complex adaptive ICT system consists of ➤ National political systems ➤ stakeholders ➤ Transportation systems ➤ technologies ➤ Immune systems ➤ threats agents ➤ The Internet ➤ policies ➤ Beehives ➤ The complexity is mostly due to: ➤ Anthills ➤ interactions between stakeholders and the networked computer system ➤ Brains ➤ communication between computers in the network ➤ ICT systems 5 6 EXAMPLES OF COMPLEX ICT SYSTEMS EXAMPLES OF STAKEHOLDERS ➤ Cloud computing infrastructures ➤ Examples of stakeholders with interest in an ICT system are ➤ Telecom infrastructures ➤ Software architects and developers ➤ Online social networks ➤ System owners, operators, and users ➤ Banking systems ➤ Governmental supervisory entities ➤ Power grids 7 8

EXAMPLES OF THREATS AGENTS COMPLEX ICT SYSTEM ➤ Benevolent users and operators making security related mistakes ➤ Insider attacks from malicious system operators Environment ➤ Outsider attacks from hackers exploiting software bugs or design flaws Policies Threats ➤ Hardware failures Observe that the stakeholders are part of the system 9 10 NEVER-ENDING CHANGE FEEDBACK LOOPS ➤ A complex adaptive ICT system’s architecture, functionality, Internal or external action technology, environment, and regulatory context change over time ➤ Complex ICT systems never reach a final form ➤ They continue to adapt to satisfy the changing needs of stakeholders and to protect against changing threats System changes System reacts ➤ A complex ICT system in “equilibrium” is a dead system 11 12

TYPES OF FEEDBACK LOOPS EXAMPLE: MALWARE EPIDEMIC ➤ A feedback loop is a series of interacting processes, which cause a system to adapt its behavior based on previous behavior ➤ It is the feedback loops that make a complex system adaptive ➤ Positive feedback loops propagate local events into global Number of 
 Deaths malware Births behavior instances ➤ Negative feedback loops dampen local events , preventing Negative 
 Positive 
 changes to global behavior feedback 
 feedback 
 loop loop 13 14 EXAMPLE: FEEDBACK IN POWER GRID POWER GRID IN EUROPE ➤ To allow for the transfer of a ship, one power line had to be Critical perturbation temporarily disconnected in Northern Germany in November ➤ Feedback loop 2006 escalates the ➤ The event triggered an overload-related cascading e ff ect and negative e ff ect many power lines went out of operation of local failure ➤ As a consequence, there were blackouts all over Europe (see ➤ Local failure black areas in picture) causes systemic failure Positive feedback loop 15 16

STOCHASTIC BEHAVIOR ➤ The behavior of a complex ICT system is modeled as a sequence of events that a ff ect a group of stakeholders both positively and negatively ➤ We consider the financial impact of all possible events during a particular time period of five to ten years ➤ The high complexity makes it necessary to represent the impact by a stochastic variable that changes with time Blackout 17 18 PROBABILITY DISTRIBUTION OF IMPACTS PROPERTIES OF IMPACT DISTRIBUTION ➤ Most of us are familiar with thin-tailed probability distributions with fixed expectation and well-defined variance ➤ The impact distribution for real-world ICT systems are likely to have ➤ time-varying expectation ➤ thick (fat) left tail ➤ infinite variance negative Impact positive 19 20

THICK LEFT TAIL PROPERTIES OF OUTLIERS ➤ Outliers are often caused by ➤ positive feedback loops that propagate local failures into systemic failures ➤ attackers exploiting software bugs and design flaws ➤ single point of failures that take down whole systems ➤ Observation Since outliers are unlikely to be in a system’s history, the past will not help us foresee outliers or calculate their probabilities Impact 21 22 EXTREME BEHAVIOR—LHR EVENT ➤ A large impact, hard-to-predict, and rare ( LHR ) event is an outlier in the left tail of the probability distribution ➤ While “normal” events occur multiple times during a period of say ten years, LHR events are non-recurrent, that is, they occur at most once during the period 23

LHR INCIDENT IN NORWEGIAN PAYMENT SERVICES LHR EVENT IN A LARGE NORWEGIAN BANK ➤ In August 2001, computer systems providing services to ➤ In March 2007, malware infected 11 000 PCs and 1 000 about one million Norwegian bank customers ceased to servers belonging to a Norwegian bank function ➤ More than two weeks were needed to completely remove the ➤ It took 7 days to get the services back in normal operation malware ➤ Multiple points of failure—causing transaction data on 288 ➤ An error in the anti-virus software and a vulnerability in the disks to become inaccessible OS led to this LHR event 25 26 LHR EVENT: CONFICKER MASSIVE RANSOMEWARE ATTACK FROM NORTH KOREA ➤ It is estimated that the Conficker worm has infected 12 ➤ Self-replicating ransomware infected 200,000 systems in more million PCs world wide than 150 countries on May 12th, 2017 ➤ Conficker severely a ff ected hospitals (Helse Vest) and the ➤ First attack to use a stolen cyberweapon developed by NSA police in Norway ➤ Many targets in Russia, Ukraine, India, and Taiwan ➤ the Norwegian police spent 30–50 million NOK to “clean ➤ 48 hospitals in Britain were a ff ected by the outbreak up” after Conficker attacked operational control centres and ➤ Renault had to stop production in some factories the system for passport control ➤ Telefónica, a Spanish telecommunications firm was a ff ected ➤ FedEx 27 28

SUMMARY OF DISCUSSION ON EXTREME BEHAVIOR Complex ICT systems are vulnerable to LHR events 30 FURTHER READING LIMITS OF RISK T E C HN I C A L I N C E RTO : L E C T U R E S N OT E S O N P R O B A B I L I TY, VO L 1 ANALYSIS SILENT RISK NASSIM NICHOLAS TALEB In which is provided a mathematical parallel version of the author’s Incerto , with derivations, examples, theorems, & heuristics. (This Draft Is For Error Detection) (OR SHIT HAPPENS IN THE FOURTH QUADRANT) 31 32

RISK IN COMPLEX ADAPTIVE ICT SYSTEMS RISK ANALYSIS ➤ We talk about risk when we do not know what will happen ➤ A classical risk analysis predicts incidents during a future time period by ➤ Risk means that more things can happen that will happen 1. describing all possible incidents, 2. estimating the probabilities that they will actually occur, and 3. determining the incidents’ impact on a group of stakeholders 33 34 CLASSICAL RISK MATRIX LIMITS OF CLASSICAL RISK ANALYSIS Is an LHR 
 low ➤ A classical risk matrix is created with the implicit assumption medium high incident a 
 that stochastic events in a system have a probability medium 
 distribution with a thin left tail risk? high ➤ LHR incidents (outliers) are ignored ➤ Observation Classical risk analysis severely underestimate the Impact medium risk associated with complex adaptive ICT systems because LHR incidents dominate the impact on stakeholders low Probability 35 36

TALEB’S FOUR QUADRANTS AVOID THE 4TH QUADRANT ➤ We want to create systems in the first quadrant but may end Impact up in the third quadrant Only local impact Global impact ➤ The important thing is to avoid the fourth quadrant with its Distribution intolerable LHR incidents 1 2 Only limited local Global impact possible Thin left tail impact but tolerable We need to develop and operate complex adaptive ICT systems that limit the impact of unforeseen incidents 3 Intolerable global 4 Large local impact impact is inevitable possible, good risk Thick left tail PREDICTIVE RISK management needed ANALYSIS DOES NOT WORK 37 38 FURTHER READING S u b j e c t i v e v i e w COMPLEXITY IS THE ENEMY 39 40