iCoq : Regression Proof Selection for Large-Scale Verification - PowerPoint PPT Presentation

iCoq : Regression Proof Selection for Large-Scale Verification Projects iCoq : Regression Proof Selection for Large-Scale Verification Projects Karl Palmskog University of Illinois at Urbana-Champaign Joint work with Ahmet Celik and Milos Gligoric at UT Austin 1 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Verification Using Proof Assistants 1 encode definitions in (higher-order) formalism 2 prove propositions interactively using powerful tactics 3 check soundness of every low-level step proof assistant tactics proof user logic engine proof checker subgoals examples: Coq, HOL4, HOL Light, Isabelle/HOL, Lean, Nuprl, ... 2 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Some Large-Scale Proof Assistant Projects Project Year Assistant Check Time LOC 4-Color Theorem 2005 Coq hours 60k Odd Order Theorem 2012 Coq hours 150k Kepler Conjecture 2015 HOL Light days 500k CompCert 2009 Coq tens of mins 40k seL4 2009 Isabelle/HOL hours 200k Verdi Raft 2016 Coq tens of mins 50k problem: long proof checking times 3 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Problem: Regression Proving in Evolving Projects Typical proving scenario: 1 change definition or lemma statement 2 begin process of re-checking all proofs 3 checking fails hours later (for seemingly unrelated proof) 4 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Problem: Regression Proving in Evolving Projects Typical proving scenario: 1 change definition or lemma statement 2 begin process of re-checking all proofs 3 checking fails hours later (for seemingly unrelated proof) Typical testing scenario: 1 change method statements or method signature 2 begin process of re-running all tests 3 testing fails hours later (for seemingly unrelated test) 4 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Regression Test Selection (RTS) A regression test selection technique chooses, from an existing test set, tests that are deemed necessary to validate modified software. Rothermel and Harrold, ACM TOSEM 6, 2 ’97 5 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Regression Test Selection (RTS) A regression test selection technique chooses, from an existing test set, tests that are deemed necessary to validate modified software. Rothermel and Harrold, ACM TOSEM 6, 2 ’97 A regression proof selection technique chooses, from an existing proof set, proofs that are deemed necessary to verify modified theories. 5 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Our Contribution 1 propose a regression proof selection (RPS) technique 2 implement RPS technique in tool, iCoq , which supports Coq 3 evaluate iCoq on revision histories of large Coq projects 6 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Coq in a Nutshell based on a constructive dependent type theory (CiC) definitions + proofs programmed in Gallina/Ltac ( .v files) coqc tool processes .v files, outputs terms ( .vo files) Require Import Alternate. Const Lemma alt_exists : forall l1 l2, exists l3, alt l1 l2 l3. Lambda Proof. induction l1; intros; destruct l2. - exists []. apply alt_nil. App App Name l1 - exists (n :: l2). apply alt_nil. - exists (a :: l1). apply alt_step. apply alt_nil. Ind list Ind nat Const list ind - specialize(IHl1 l2). destruct IHl1. exists (a :: n :: x). repeat apply alt_step. auto. Qed. AltLem.v AltLem.vo 7 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Coq v8.5 Asynchronous Proof-Checking Toolchain newly-added toolchain can produce .vio files without proofs .vio files contain proof tasks, checked asynchronously alt exists AltLem.v proof script coqc -quick alt exists AltLem.vio proof task coqc -check-vio-tasks 8 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Regression Proof Selection Technique Three phases: 1 analysis : locate proofs affected by changes 2 execution : emit and run proof-checking commands 3 collection : find dependencies of modified definitions and lemmas+proofs Key idea: maintain file and identifier dependency graphs 9 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Example, revision 1 Require Export List. Export ListNotations. Require Import Alternate. Fixpoint alternate l1 l2 : list nat := Lemma alt_exists : match l1 with forall l1 l2, exists l3, alt l1 l2 l3. | [] ⇒ l2 | h1 :: t1 ⇒ Proof. match l2 with induction l1; intros; destruct l2. | [] ⇒ h1 :: t1 - exists []. apply alt_nil. | h2 :: t2 ⇒ - exists (n :: l2). apply alt_nil. h1 :: h2 :: alternate t1 t2 - exists (a :: l1). apply alt_step. end apply alt_nil. end. - specialize(IHl1 l2). destruct IHl1. exists (a :: n :: x). Inductive alt : list nat → list nat → repeat apply alt_step. auto. list nat → Prop := Qed. | alt_nil : forall l, alt [] l l AltLem.v | alt_step : forall a l t1 t2, alt l t1 t2 → alt (a :: t1) l (a :: t2). Lemma alt_alternate : forall l1 l2 l3, alt l1 l2 l3 → alternate l1 l2 = l3. Proof. (* ... omitted proof script ... *) Qed. Alternate.v 10 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Example, revision 1 Require Export List. Export ListNotations. Require Import Alternate. Fixpoint alternate l1 l2 : list nat := Lemma alt_exists : match l1 with forall l1 l2, exists l3, alt l1 l2 l3. | [] ⇒ l2 | h1 :: t1 ⇒ Proof. match l2 with induction l1; intros; destruct l2. | [] ⇒ h1 :: t1 - exists []. apply alt_nil. | h2 :: t2 ⇒ - exists (n :: l2). apply alt_nil. h1 :: h2 :: alternate t1 t2 - exists (a :: l1). apply alt_step. end apply alt_nil. end. - specialize(IHl1 l2). destruct IHl1. exists (a :: n :: x). Inductive alt : list nat → list nat → repeat apply alt_step. auto. list nat → Prop := Qed. | alt_nil : forall l, alt [] l l AltLem.v | alt_step : forall a l t1 t2, alt l t1 t2 → alt (a :: t1) l (a :: t2). Lemma alt_alternate : Alternate.v List.v forall l1 l2 l3, alt l1 l2 l3 → alternate l1 l2 = l3. Proof. (* ... omitted proof script ... *) AltLem.v Qed. File dependency graph Alternate.v 10 / 21

iCoq : Regression Proof Selection for Large-Scale Verification Projects Example, revision 1 Require Export List. Export ListNotations. Require Import Alternate. Fixpoint alternate l1 l2 : list nat := Lemma alt_exists : match l1 with forall l1 l2, exists l3, alt l1 l2 l3. | [] ⇒ l2 | h1 :: t1 ⇒ Proof. match l2 with induction l1; intros; destruct l2. | [] ⇒ h1 :: t1 - exists []. apply alt_nil. | h2 :: t2 ⇒ - exists (n :: l2). apply alt_nil. h1 :: h2 :: alternate t1 t2 - exists (a :: l1). apply alt_step. end apply alt_nil. end. - specialize(IHl1 l2). destruct IHl1. exists (a :: n :: x). Inductive alt : list nat → list nat → repeat apply alt_step. auto. list nat → Prop := Qed. | alt_nil : forall l, alt [] l l AltLem.v | alt_step : forall a l t1 t2, alt l t1 t2 → alt (a :: t1) l (a :: t2). # coqc -quick Alternate.v Lemma alt_alternate : forall l1 l2 l3, alt l1 l2 l3 → # coqc -quick AltLem.v alternate l1 l2 = l3. Proof. # coqc -check-vio-tasks 0 Alternate.vio (* ... omitted proof script ... *) Qed. # coqc -check-vio-tasks 0 AltLem.vio Alternate.v 10 / 21

iCoq : Regression Proof Selection for Large-Scale Verification - PowerPoint PPT Presentation

iCoq : Regression Proof Selection for Large-Scale Verification Projects iCoq : Regression Proof Selection for Large-Scale Verification Projects Karl Palmskog University of Illinois at Urbana-Champaign Joint work with Ahmet Celik and Milos

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Karl Palmskog , Ahmet

Least Squares Support Vector Regression with Applications to Large-Scale Data: a Statistical

Joint Feature Selection in Distributed Stochastic Learning for Large-Scale Discriminative SMT

STARTS: STARTS: STARTS: STARTS: STAtic STAtic Regression Test Selection Regression Test

Optimal Large-Scale Internet Media Selection Gareth James Department of Data Sciences and

FINANCING LARGE SCALE SOLAR Large Scale Solar Conference - Sydney Gloria Chan Director, Large

Large scale greedy feature-selection for multi-target learning Antti Airola, Tapio Pahikkala et

Quantile Regression for Large-scale Applications Jiyan Yang Stanford University June 19, 2013

Resources Description, Selection, Reservation and Verification on a Large-scale Testbed David

Formalizing Mathematics-In Praxis: First experiences with Isabelle/HOL ALEXANDRIA: Large-Scale

INCORPORATING LARGE-SCALE CITIZEN INCORPORATING LARGE-SCALE CITIZEN DELIBERATION INTO

Large-Scale Survey Interviewing Following Large Scale Survey Interviewing Following the 2008

Regression : feat u re selection P R AC TIC IN G MAC H IN E L E AR N IN G IN TE R VIE W QU E

Large-Scale Machine Learning Jean-Philippe Vert jean-philippe.vert@ { mines-paristech,curie,ens }

Lecture 6: Multiple Linear Regression, Polynomial Regression and Model Selection CS109A

Large-scale production of graphene: Introduction Large-scale production of graphene: what is

using R for regression model selection with adaptive penalties procedures based on the FDR

Efficient Interactive Training Selection for Large-scale Entity Resolution Qing Wang, Dinusha

Ethics in Techniques for large-scale data Graham J.L. Kemp TECHNIQUES FOR LARGE-SCALE DATA

LS-SVMlab & Large scale modeling Kristiaan Pelckmans, ESAT- SCD/SISTA J.A.K. Suykens, B. De

Towards Refactoring-Aware Regression Test Selection Kaiyuan Wang , Chenguang Zhu, Ahmet Celik,

Practical Regression Test Selection with Dynamic File Dependencies Milos Gligoric, Lamyaa

An Email Contact Protocol Experiment in a Large-Scale Survey of U.S. in a Large Scale Survey of

Ridge/Lasso Regression, Model selection Xuezhi Wang Computer Science Department Carnegie Mellon

iCoq : Regression Proof Selection for Large-Scale Verification - PowerPoint PPT Presentation

iCoq : Regression Proof Selection for Large-Scale Verification Projects iCoq : Regression Proof Selection for Large-Scale Verification Projects Karl Palmskog University of Illinois at Urbana-Champaign Joint work with Ahmet Celik and Milos

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Karl Palmskog , Ahmet

Least Squares Support Vector Regression with Applications to Large-Scale Data: a Statistical

Joint Feature Selection in Distributed Stochastic Learning for Large-Scale Discriminative SMT

STARTS: STARTS: STARTS: STARTS: STAtic STAtic Regression Test Selection Regression Test

Optimal Large-Scale Internet Media Selection Gareth James Department of Data Sciences and

FINANCING LARGE SCALE SOLAR Large Scale Solar Conference - Sydney Gloria Chan Director, Large

Large scale greedy feature-selection for multi-target learning Antti Airola, Tapio Pahikkala et

Quantile Regression for Large-scale Applications Jiyan Yang Stanford University June 19, 2013

Resources Description, Selection, Reservation and Verification on a Large-scale Testbed David

Formalizing Mathematics-In Praxis: First experiences with Isabelle/HOL ALEXANDRIA: Large-Scale

INCORPORATING LARGE-SCALE CITIZEN INCORPORATING LARGE-SCALE CITIZEN DELIBERATION INTO

Large-Scale Survey Interviewing Following Large Scale Survey Interviewing Following the 2008

Regression : feat u re selection P R AC TIC IN G MAC H IN E L E AR N IN G IN TE R VIE W QU E

Large-Scale Machine Learning Jean-Philippe Vert jean-philippe.vert@ { mines-paristech,curie,ens }

Lecture 6: Multiple Linear Regression, Polynomial Regression and Model Selection CS109A

Large-scale production of graphene: Introduction Large-scale production of graphene: what is

using R for regression model selection with adaptive penalties procedures based on the FDR

Efficient Interactive Training Selection for Large-scale Entity Resolution Qing Wang, Dinusha

Ethics in Techniques for large-scale data Graham J.L. Kemp TECHNIQUES FOR LARGE-SCALE DATA

LS-SVMlab &amp; Large scale modeling Kristiaan Pelckmans, ESAT- SCD/SISTA J.A.K. Suykens, B. De

Towards Refactoring-Aware Regression Test Selection Kaiyuan Wang , Chenguang Zhu, Ahmet Celik,

Practical Regression Test Selection with Dynamic File Dependencies Milos Gligoric, Lamyaa

An Email Contact Protocol Experiment in a Large-Scale Survey of U.S. in a Large Scale Survey of

Ridge/Lasso Regression, Model selection Xuezhi Wang Computer Science Department Carnegie Mellon

LS-SVMlab & Large scale modeling Kristiaan Pelckmans, ESAT- SCD/SISTA J.A.K. Suykens, B. De