Formal verification of a realistic compiler Xavier Leroy CACM 2009 - PowerPoint PPT Presentation

Formal verification of a realistic compiler Xavier Leroy CACM 2009 CS 7194: Great Works in Programming Languages Presenter : Irene Yoon | Mentor : Ryan Doenges � 1

Building robust compilers is Hard. � 2

Bugs bugs � 3

Bugs [Yang et al 2011] • Random testing finds bugs in 11 C compilers • Hundreds of previously unknown bugs • LLVM has a large test suite, real compilers have bugs � 4

Bugs [Yang et al 2011] • Random testing finds bugs in 11 C compilers • Hundreds of previously unknown bugs • LLVM has a large test suite � 5

Bugs [Yang et al 2011] • Random testing finds bugs in 11 C compilers • Hundreds of previously unknown bugs • LLVM has a large test suite, real compilers have bugs � 6

Bugs [Yang et al 2011] • Random testing finds bugs in 11 C compilers • Hundreds of previously unknown bugs • LLVM has a large test suite � 7

✅ Building compilers is hard Testing sucks Formalisms are good � 8

✅ Building compilers is hard ✅ Testing sucks Formalisms are good � 9

✅ Building compilers is hard ✅ Testing sucks ✅ Formalisms are good � 10

✅ Building compilers is hard ✅ Testing sucks ✅ Formalisms are good Formal verification of a compiler � 11

First Published Proof of Compiler Correctness [1967] • arithmetic expressions → stack machine code • prototype for proving usable compilers � 12

First Mechanized Proof of Compiler Correctness [1972] • ALGOL-like language → elementary assembly language • Stanford LCF � 13

Compiler Verification • 100+ papers on compiler verification since 1967 � 14

Compiler Verification • 100+ papers on compiler verification since 1967 � 15

Compiler Verification • 100+ papers on compiler verification since 1967 � 16

Compiler Verification • 100+ papers on Compiler Verification since 1967 [2003] � 17

CompCert � 18

CompCert [2009] “Develop and prove correct a realistic compiler, usable for critical embedded software.” • 42k Coq, 3 person years target source PowerPC Clight ARM x86 CompCert

CompCert [2009] “Develop and prove correct a realistic compiler, usable for critical embedded software.” • 42k Coq, 3 person years target source PowerPC Clight ARM x86 CompCert

CompCert [2009] “Develop and prove correct a realistic compiler, usable for critical embedded software.” • 42k Coq, 3 person years target source PowerPC Clight ARM x86 CompCert

Verified, Validated, Certifying � 22

Verified, Validated, Certifying 1. Verified transformation [ Compiler Correctness ] source target COMPILER � 23

Verified, Validated, Certifying 2 . Translation validation [ Translation Verification ] target source COMPILER VALIDATOR � 24

Verified, Validated, Certifying 3. Certifying compiler [ Proof-carrying Code ] source CERTIFYING PROOF COMPILER CHECKER target code + certificate � 25

� 26

[Leroy `06] � 27

=> External solver with verified validation � 28

=> External solver with verified validation [Tristan and Leroy `10] � 29

CompCert � 30

CompCert formal specification � 31

CompCert formal specification → semantic analysis tools [Appel ’11] � 32

CompCert semantic preservation* � 33

CompCert � 34

Semantic Preservation • Spec(B) : functional specification of observable behavior • B: observable behavior (trace properties of I/O) • “going wrong” (run-time error), termination, divergence • C ㅑ Spec if A. C cannot go wrong B. All behaviors B satisfy Spec � 35

Semantic Preservation • Spec(B) : functional specification of observable behavior • B: observable behavior (trace properties of I/O) • “going wrong” (run-time error), termination, divergence • C ㅑ Spec if A. C cannot go wrong B. All behaviors B satisfy Spec � 36

Semantic Preservation • Spec(B) : functional specification of observable behavior • B: observable behavior (trace properties of I/O) • “going wrong” (run-time error), termination, divergence • C ㅑ Spec if A. C cannot go wrong B. All behaviors B satisfy Spec � 37

Correctness Property Compiled code C preserves the fact that the source code S satisfies the specification. � 38

Proving Semantic Preservation � 39

Proving Semantic Preservation * � 40

Safety Precondition • Compilation result will match the semantics of the input if if program is “safe” (no runtime errors) • Need to prove that input program is safe � 41

Safety Precondition • Compilation result will match the semantics of the input if if program is “safe” (no runtime errors) • Need to prove that input program is safe � 42

Correctness Weakness � 43

Correctness Weakness � 44

CompCert � 45

CompCert � 46

Correctness Weakness • Only runs after the preprocessing step • Astrèe [Cousot et al ’05] , Verasco [Jourdan et al ’15] ) • Reliant on less verifiable assumptions • Coq’s correctness [Anand et al ’17] • Formal specification of C & PowerPC assembly � 47

Correctness Weakness • Only runs after the preprocessing step • Astrèe [Cousot et al ’05] , Verasco [Jourdan et al ’15] • Reliant on less verifiable assumptions • Coq’s correctness [Anand et al ’17] • Formal specification of C & PowerPC assembly � 48

Correctness Weakness • Only runs after the preprocessing step • Astrèe [Cousot et al ’05] , Verasco [Jourdan et al ’15] ) • Reliant on less verifiable assumptions • Coq’s correctness [Anand et al ’17] • Formal specification of C & PowerPC assembly � 49

Correctness Weakness • Only runs after the preprocessing step • Astrèe [Cousot et al ’05] , Verasco [Jourdan et al ’15] ) • Reliant on less verifiable assumptions • Coq’s correctness ( CertiCoq [Anand et al ’17]) • Formal specification of C & PowerPC assembly � 50

Correctness Weakness • Only runs after the preprocessing step • Astrèe [Cousot et al ’05] , Verasco [Jourdan et al ’15] ) • Reliant on less verifiable assumptions • Coq’s correctness ( CertiCoq [Anand et al ’17]) • Formal specification of C & PowerPC assembly � 51

Performance � 52

Performance competitive with gcc -01 � 53

Bugs, revisited. [Yang et al 2011] • CompCert: errors only found in unverified parts (parser and model of machine) • Other compilers: errors everywhere “The striking thing about our CompCert results is that the middle-end bugs we found in all other compilers are absent” � 54

Bugs, revisited. [Yang et al 2011] • CompCert: errors only found in unverified parts (parser and model of machine) • Other compilers: errors everywhere “The striking thing about our CompCert results is that the middle-end bugs we found in all other compilers are absent” � 55

Bugs, revisited. [Yang et al 2011] • CompCert: errors only found in unverified parts (parser and model of machine) • Other compilers: errors everywhere “The striking thing about our CompCert results is that the middle-end bugs we found in all other compilers are absent” � 56

Bugs, revisited. [Yang et al 2011] • CompCert: errors only found in unverified parts (parser and model of machine) • Other compilers: errors everywhere “The striking thing about our CompCert results is that the middle-end bugs we found in all other compilers are absent” � 57

Critical Use Cases • AirBus • MTU Friedrichshafen (nuclear energy) • High-Assurance Cyber Military Systems (HACMS) [Fisher et al, ’17] • PhD Theses � 58

Critical Use Cases • AirBus • MTU Friedrichshafen (nuclear energy) • High-Assurance Cyber Military Systems (HACMS) [Fisher et al, ’17] • PhD Theses � 59

Critical Use Cases • AirBus • MTU Friedrichshafen (nuclear energy) • High-Assurance Cyber Military Systems (HACMS) [Fisher et al, ’17] • PhD Theses � 60

Critical Use Cases • AirBus • MTU Friedrichshafen (nuclear energy) • High-Assurance Cyber Military Systems (HACMS) [Fisher et al, ’17] • PhD Theses � 61

Critical Use Cases “a realistic compiler” • AirBus • MTU Friedrichshafen (nuclear energy) • High-Assurance Cyber Military Systems (HACMS) [Fisher et al, ’17] • PhD Theses � 62

Concluding Remarks • Still some correctness and safety weaknesses • Useful for safety critical code (that doesn’t have to run fast) • Future work - . . . � 63

Concluding Remarks • Still some correctness and safety weaknesses • Useful for safety critical code (that doesn’t have to run fast) • Future work - . . . . . . � 64

Concluding Remarks • Still some correctness and safety weaknesses • Useful for safety critical code (that doesn’t have to run fast) • Future work - . . . � 65

Concluding Remarks • Still some correctness and safety weaknesses • Useful for safety critical code (that doesn’t have to run fast) Type Preserving Compilation • Future work - . . . . . . � 66