Towards Automated Safety Vetting of PLC Code in Real-World Plants Mu - PowerPoint PPT Presentation

Towards Automated Safety Vetting of PLC Code in Real-World Plants Mu Zhang ∗ , Chien-Ying Chen†, Bin-Chou Kao‡, Yassine Qamsane§, Yuru Shao¶, Yikai Lin¶, Elaine Shi ∗ , Sibin Mohan†, Kira Barton§, James Moyne§ and Z. Morley Mao¶ ∗ CS, Cornell; †CS, UIUC; ‡ITI, UIUC; §ME, UMich; ¶EECS, UMich ∗ mz496@cornell.edu, ∗ elaine@cs.cornell.edu, †{cchen140,sibin}@illinois.edu, ‡ bkao2@illinois.edu, §{yqamsane,bartonkl,moyne}@umich.edu, ¶{yurushao,yklin,zmao}@umich.edu

Safety Hazards are Unique Threats in ICS 2

PLC being a Major Attack Vector Programmable Physical Damage Controller Code w/ Logic Controller Safety Violations (PLC) Insider Attacks or Bugs Core Control Unit on Di ff erent from Financial the Factory Floor Loss Often Seen in Attacks in Consumer Systems A great many of prior work: e.g., TSV (NDSS’14), S YM PLC (FSE’17) 3

Overlooked Fact: ICS is Complex; PLC is NOT Working Alone CNC PLC Real-world Automotive Manufacturing Testbed Computer Numerical Programmable Logic Controller Control Machine Developed by No.1 Vendor Robot (Rockwell Automation) PLCs are driven by events from other machines Robot Part (Vehicle Frame) Testing PLC code on Pallet requires external event inputs 4

Testing Event-driven Code in Other Domains 
 – Simulating and Rearranging Events Web Program: SymJS FSE’14 , Saxena Oakland’10 Android App: Anand FSE’12 , Jensen ISSTA’13 , Mirzaei Softw. Eng. Notes’12 , Yang CCS’13 Simulated Event Sequence Crash Rearrange Event Order App Testing in Emulator … 5

is NOT Su ffi cient Rearranging Event Order to Test PLC Code 10s 7s PLC Simulator Event Sequences of Same Ordering But Di ff erent Timings Timing factor: Nature of ICS Timeliness, Throughput à Internal Timeouts Machine Speed Limits à External Timing Constraints 6

A Running Example Events Received by PLC Deliver Part Update Part Pallet { time 0.5s Update_Complete = TRUE Update_Complete = TRUE && Part_AtConveyor = TRUE && Part_AtConveyor = TRUE Safety Req: <= 30s X Pallet enters Pallet leaves Violated TPTL Spec: 7

Traditional Event Permutation Doesn’t Solve the Problem 1->….->5-> 6->7 Correct! 5-> 7->6 Error! 5-> 7->6 Still Correct! 0.5s 0.5s 0.5s …. …. …. time time time 8

V ET PLC: Generating Timed Event Sequences to enable Automated Safety Vetting of PLC Code Program Analysis on PLC/Robot: 30s 1m 10s 45s Generating Event Causality Graph PLC Simulator Execution Traces Safety Violations Timed Event Sequences Data Mining on Runtime Data: Discovering Temporal Invariants 9

V ET PLC on Running Example PLC FANUC Robot DI[0] -> PICKCNC1 IF(NOT Part_AtConveyor) PICKCNC1 THEN DI[0]=TRUE … … L P[0] 100mm/sec FINE IF(Update_Complete) … THEN … DO[2:CNC1 part@conveyor]=ON … WAIT .50(sec) IF(Part_AtConveyor) IF(Part_AtConveyor) DO[2:CNC1 THEN … THEN … { { part@conveyor]=OFF time ​𝑬𝒋𝒕𝒖 𝒕𝒖𝒃𝒐𝒅 𝒐𝒅𝒇/​𝑻𝒒 𝑻𝒒𝒇𝒇𝒆 𝒇𝒇𝒆↓𝑺 ↓𝑺𝒑𝒄𝒑𝒖 = 𝐄𝐟𝐦𝐣𝐰 𝐄𝐟𝐦𝐣𝐰𝐟𝐬𝐳 𝐟𝐬𝐳𝐔𝐣𝐧𝐟 𝐔𝐣𝐧𝐟 Timeout { Constant (0.5s) Soft Invariant in Robot Code – Can be derived from Update I/O Time testbed: Speed x Time Soft Timing Invariant Configurable - Can be observed from testbed Variable 10

Timed Event Causality Graph (TECG): Find Valid Event Orders PLC Side Pallet_Sensor ¬ Part_Sensor P_IN, (P) P_IN, (P) Pallet_Arrival Robot Side P_Local, (P) CNC_Part_Ready Event Name Part_AtConveyor P_IN, (P) Type, (Duration) DI[0] P_IN, (0.5s) Update_Part_Process R_IN, (P) Robot_Ready P_Local, (P) P_IN, (P) Deliver_Part [15s, 20s] ¬ P_OUT, (P) [3s, 39.4s] Part_AtConveyor DO[2] P_IN, (P) R_OUT, (0.5s) RFID_IO_Complete P_IN, (P) Context-Sensitive, Flow sensitive, Update_Complete P_Local, (P) Inter-procedural Dataflow Analysis 11

Mining Temporal Invariants for Events: 2 Steps Step 1 : Qualitative “followed-by”: Follows[ ε a ][ ε b ] = Occurrence[ ε a ] – Synoptic ( FSE’11 ) Step 2 : Quantitative “with-in”: t x .( ε a → t y .( ε b ∧ t y − t x ≥ τ lower )) – Perfume ( ASE’14 ) t x .( ε a → t y .( ε b ∧ t y − t x ≤ τ upper )) Results for Motivating Example (1.2 GB data for 10 hours): Advantage of TECG: Only need to mine relations that do not contradict TECG 12

Safety Violation Triggered Creating Timed Event Sequences How to discretize durations? Pallet_Sensor ¬ Part_Sensor x CNC_Part_Ready x Robot_Ready x ¬ Part_AtConveyor Update_Complete Part_AtConveyor T+10 0.5s Part_AtConveyor 13

Evaluation on Real Testbeds for Di ff erent Scenarios 2 Di ff erent CNC Testbeds Robot Robot PLC SMART : Automotive Production Line Fischertechnik : Part Processing w/ 4 PLCs 10 Safety-critical S1: Conveyor Overflow #1 S6: Ram-Part Collision S2: Robot in Danger Zone S7: CNC-Part Collision Scenarios S3: Conveyor Overflow #2 S8: Conveyor Overflow #3 S4: Part-Gate Collision S9: Conveyor Underflow S5: CNC Overflow S10: Ram-Part Collision #2 14

Evaluation: How many sequences are created? 90000 80000 70000 Untimed AllSeqs 60000 Untimed VetPLC-Seqs 50000 VetPLC-TSeqs-2 (Coarse) 40000 VetPLC-TSeqs-5(Medium) 30000 VetPLC-TSeqs-10(Fine) 20000 10000 0 1 2 3 4 5 6 7 8 9 10 Red à Green: Program analysis reduces amount of event sequences Green à Orange à Black à Blue: Time discretization can significantly increases that 15

Bug Detected? State-of-the-Art vs. V ET PLC V ET PLC State-of-the-art More Time Slices -> More Precise Error-Triggering Range V ET PLC Outperforms State-of-the-art! Empirically, 5 slices works better. 16

Conclusion q Insight: r eal-world PLC code is event-driven and timing-sensitive q Solution: V ET PLC automatically constructs timed event sequences via analyzing event causalities in PLC/robot code plus mining runtime data from physical testbeds q E ff ectiveness: V ET PLC outperforms state-of-the-art and has found “organic” vulnerabilities in two di ff erent types of real-world ICS testbeds. 17

Thank you!

PLC Programming Paradigm: Scan Cycle Input Phase v No dataflow in one cycle v Dataflow IF Pallet_Sensor AND NOT (Part_Sensor) THEN across cycles Pallet_Arrival_NEW Pallet_Arrival := true; v Any “Define” in END_IF; a cycle may X IF Part_Sensor THEN a ff ect “Use” in Computation Phase Retract_Stopper := true; the next END_IF; Pallet_Arrival_OLD IF Pallet_Arrival AND … THEN Deilver_Part := true; … END_IF; Output Phase Pallet_Arrival_OLD := Pallet_Arrival_NEW 19

Technical Challenge: Distributed Event Sources [24.4s, 24.6s] Reality Ideally PLC E Deliver_Part E Deliver_Part E Part_AtConveyor Robot E Part_AtConveyor Solution: Inferring Events from State Variables D Deliver_Part D Part_AtConveyor 20

Speed Reconfiguration ∵ 𝒑𝒄 = ​𝑘𝑝𝑐/​𝑡𝑞𝑓𝑓𝑒↓𝑑𝑝𝑜𝑔 ≤ ​𝜐↓𝑣𝑞𝑞𝑓𝑠 Time variation caused by physical ​𝜐↓𝑚𝑝𝑥𝑓𝑠 ≤ ​𝑼↓ 𝑼↓𝒌𝒑𝒄 operations or program execution paths 𝒕𝒒𝒇𝒇𝒆↓𝒏𝒃𝒚 Time variation caused by ​𝒕𝒒 𝒕𝒒𝒇𝒇𝒆↓𝒏𝒋𝒐 𝒐 ≤ ​𝑡𝑞𝑓𝑓𝑒↓𝑑𝑝𝑜𝑔 ≤ ​𝒕𝒒 reconfiguring machine speeds ∴ 𝒕𝒒𝒇𝒇𝒆↓𝒏𝒋𝒐 ​𝜐↓𝑚𝑝𝑥𝑓𝑠 × ​𝑡𝑞𝑓𝑓𝑒↓𝑑𝑝𝑜𝑔 /​𝒕𝒒 𝒕𝒒𝒇𝒇𝒆↓𝒏𝒃𝒚 ≤ ​𝑼↓ 𝑼↓𝒌𝒑𝒄 𝒑𝒄 ≤ ​𝜐↓𝑣𝑞𝑞𝑓𝑠 × ​𝑡𝑞𝑓𝑓𝑒↓𝑑𝑝𝑜𝑔 /​𝒕𝒒 Speed rated 0 ? Speed high-throughput ? Speed high-throughput-and-safe 21

Scenario-Specific Safety Specs 22