Proving Unrealizability for Syntax-Guided Synthesis Qinheping Hu , - PowerPoint PPT Presentation

Proving Unrealizability for Syntax-Guided Synthesis Qinheping Hu , Jason Breck , John Cyphert , Loris D'Antoni , Thomas Reps

Proving Unrealizability for Syntax-Guided Synthesis 1

Syntax-Guided Synthesis (SyGuS) Specification 𝜒 𝑛𝑏𝑦(𝑦, 𝑧), 𝑦, 𝑧 : 𝑛𝑏𝑦 𝑦, 𝑧 ≥ 𝑦 Solution Program SyGuS ∧ 𝑛𝑏𝑦 𝑦, 𝑧 ≥ 𝑧 𝑓 ∈ 𝑀(𝐻) such that Solver ∧ (𝑛𝑏𝑦 𝑦, 𝑧 = 𝑦 ∨ 𝑛𝑏𝑦 𝑦, 𝑧 = 𝑧) ∀𝑦, 𝑧. 𝜒 𝑓, 𝑦, 𝑧 Search space 𝐻 : max 𝑦, 𝑧 = 𝐽𝑈𝐹(> (𝑦, 𝑧), 𝑦, 𝑧) Start → +(Start, Start) | 𝐽𝑈𝐹(BExpr, Start, Start) 𝑦 𝑧 0 1 BExpr → 𝑂𝑝𝑢(BExpr ) | > (Start, Start) |𝐵𝑜𝑒(BExpr, BExpr)

Syntax-Guided Synthesis (SyGuS) • Goal: find a program 𝑓 ∈ 𝑀(𝐻) such that ∀𝑦, 𝑧. 𝜒 𝑓, 𝑦, 𝑧 • SyGuS-Competition • SyGuS Solvers: CVC4, EUSolver, Euphony, DryadSynth, LoopInvGen, E3Solver, Esolver What if there doesn’t exist 𝑓 ∈ 𝑀(𝐻) such that ∀𝑦, 𝑧. 𝜒 𝑓, 𝑦, 𝑧 (Unrealizable)

2 Proving Unrealizability for Syntax-Guided Synthesis 1

Example of Unrealizable SyGuS Problems Specification ∀𝑦, 𝑧. max 𝑦, 𝑧 ≥ 𝑦 ∧ max 𝑦, 𝑧 ≥ 𝑧 ∧ (max 𝑦, 𝑧 = 𝑦 ∨ max 𝑦, 𝑧 = 𝑧) Search space BExpr = 𝑂𝑝𝑢(BExpr ) Start = +(Start, Start) | 𝐽𝑈𝐹(BExpr, Start, Start) | > (Start, Start) 𝑦 𝑧 0 1 |𝐵𝑜𝑒(BExpr, BExpr) max 𝑦, 𝑧 = 𝐽𝑈𝐹(> (𝑦, 𝑧), 𝑦, 𝑧)

Example of Unrealizable SyGuS Problems Specification ∀𝑦, 𝑧. max 𝑦, 𝑧 ≥ 𝑦 ∧ max 𝑦, 𝑧 ≥ 𝑧 ∧ (max 𝑦, 𝑧 = 𝑦 ∨ max 𝑦, 𝑧 = 𝑧) Search space Start = +(Start, Start) 𝑦 𝑧 0 1 No Solution

3 2 Proving Unrealizability for Syntax-Guided Synthesis 1

φ: 𝑔 𝑦, 𝑧 ≥ 𝑦 ∧ 𝑔 𝑦, 𝑧 ≥ 𝑧 ∧ (𝑔 𝑦, 𝑧 = 𝑦 ∨ 𝑔 𝑦, 𝑧 = 𝑧) 𝑇𝑧 ≔ G: Start = +(Start, Start) x 𝑧 0 1 CEGIS 𝑇𝑧 𝐹 Examples E: Synthesizer (x0,y0)=(0,0) Verifier

𝑇𝑧 𝐹 : ∧ 𝑦,𝑧 ∈𝐹 φ(𝑔, 𝑦, 𝑧) 𝑇𝑧 𝐹 Examples E: Synthesizer (x0,y0)=(0,0)

φ: 𝑔 𝑦, 𝑧 ≥ 𝑦 ∧ 𝑔 𝑦, 𝑧 ≥ 𝑧 ∧ (𝑔 𝑦, 𝑧 = 𝑦 ∨ 𝑔 𝑦, 𝑧 = 𝑧) 𝑇𝑧 ≔ G: Start = +(Start, Start) x 𝑧 0 1 CEGIS 𝑇𝑧 𝐹 Examples E: Synthesizer (x0,y0)=(0,0) 𝑔 𝑦, 𝑧 = 0 Verifier new ce (0,1)

φ: 𝑔 𝑦, 𝑧 ≥ 𝑦 ∧ 𝑔 𝑦, 𝑧 ≥ 𝑧 ∧ (𝑔 𝑦, 𝑧 = 𝑦 ∨ 𝑔 𝑦, 𝑧 = 𝑧) 𝑇𝑧 ≔ G: Start = +(Start, Start) x 𝑧 0 1 CEGIS 𝑇𝑧 𝐹 Examples E: Synthesizer (x0,y0)=(0,0) (x1,y1)=(0,1) 𝑔 𝑦, 𝑧 = 𝑧 Verifier new ce (1,0)

φ: 𝑔 𝑦, 𝑧 ≥ 𝑦 ∧ 𝑔 𝑦, 𝑧 ≥ 𝑧 ∧ (𝑔 𝑦, 𝑧 = 𝑦 ∨ 𝑔 𝑦, 𝑧 = 𝑧) 𝑇𝑧 ≔ G: Start = +(Start, Start) x 𝑧 0 1 CEGIS 𝑇𝑧 𝐹 Examples E: Synthesizer (x0,y0)=(0,0) (x1,y1)=(0,1) 𝑔 𝑦, 𝑧 = 1 (x2,y2)=(1,0) Verifier new ce (2,0)

φ: 𝑔 𝑦, 𝑧 ≥ 𝑦 ∧ 𝑔 𝑦, 𝑧 ≥ 𝑧 ∧ (𝑔 𝑦, 𝑧 = 𝑦 ∨ 𝑔 𝑦, 𝑧 = 𝑧) 𝑇𝑧 ≔ G: Start = +(Start, Start) x 𝑧 0 1 CEGIS 𝑇𝑧 𝐹 Examples E: Unrealizable ! Synthesizer (x0,y0)=(0,0) (x1,y1)=(0,1) (x2,y2)=(1,0) (x3,y3)=(2,0) Verifier

𝑡𝑧 𝐹 𝑡𝑧 is unrealizable is unrealizable No solution over all inputs No solution over 𝐹

From SyGuS over Examples to a Reachability Problem

Reachability Problem Non-deterministic choice void main(){ Reachability solver: int x = 0; CPA-checker while(nd()){ Uautomizer x++; Seahorn … } assert(x<0) } Goal : can the assert be falsified?

Overview SyGuS over examples Reachability problem 𝑡𝑧 𝐹 𝑆𝑓 𝐹 void main(){ (φ 𝐹 , 𝐻) … assert (…)} 𝑡𝑧 𝐹 is unrealizable assert cannot be falsified

𝑇𝑧 𝐹 to 𝑆𝑓 𝐹 Set input to 𝐹 𝑦 ← 𝐹 Ԧ 𝐻 is non-deterministically drawn from 𝑀(𝐻) 𝑔 𝑝 ← 𝑔 Ԧ 𝐻 ( Ԧ 𝑦) Check if Ԧ 𝑝 doesn’t satisfy φ 𝑔 𝐻 ( Ԧ 𝑦) satisfy φ on 𝐹 assert (¬ٿ 𝑦 𝑗 ∈ 𝐹. φ(𝑝 𝑗 , 𝑦 𝑗 )) 𝑇𝑧 𝐹 is unrealizable

Set input to 𝐹 𝑦 ← 𝐹 Ԧ Examples E: (x0,y0)=(0,0) x0 = 0; (x1,y1)=(0,1) y0 = 0; x1 = 0; y1 = 1;

𝑇𝑧 𝐹 to 𝑆𝑓 𝐹 Set input to 𝐹 𝑦 ← 𝐹 Ԧ 𝐻 is non-deterministically drawn from 𝑀(𝐻) 𝑔 𝑝 ← 𝑔 Ԧ 𝐻 ( Ԧ 𝑦) Check if Ԧ 𝑝 doesn’t satisfy φ assert (¬ٿ 𝑦 𝑗 ∈ 𝐹. φ(𝑝 𝑗 , 𝑦 𝑗 ))

Check if Ԧ 𝑝 doesn’t satisfy φ assert (¬ٿ 𝑦 𝑗 ∈ 𝐹. φ(𝑝 𝑗 , 𝑦 𝑗 )) void main(){ … assert(!(spec(x0,y0,o0)&&spec(x1,y1,o1))); } bool spec(x,y,o){ return (o>=x)&&(o>=y)&&(o==x||o==y); } φ 𝑔 𝑦, 𝑧 ≔ 𝑔 𝑦, 𝑧 ≥ 𝑦 ∧ 𝑔 𝑦, 𝑧 ≥ 𝑧 ∧ (𝑔 𝑦, 𝑧 = 𝑦 ∨ 𝑔 𝑦, 𝑧 = 𝑧)

𝑇𝑧 𝐹 to 𝑆𝑓 𝐹 Set input to 𝐹 𝑦 ← 𝐹 Ԧ 𝐻 is non-deterministically drawn from 𝑀(𝐻) 𝑔 𝑝 ← 𝑔 Ԧ 𝐻 ( Ԧ 𝑦) Check if Ԧ 𝑝 doesn’t satisfy φ assert (¬ٿ 𝑦 𝑗 ∈ 𝐹. φ(𝑝 𝑗 , 𝑦 𝑗 ))

𝐻 is non-deterministically drawn from 𝑀(𝐻) 𝑔 𝑝 ← 𝑔 Ԧ 𝐻 ( Ԧ 𝑦) o0 = fStart(x0,y0); int fStart(x0,y0){ \\ Start -> 0 if(nd()){ return 0;} \\ Start -> 1 if(nd()){ return 1;} \\ Start -> x if(nd()){ return x0;} \\ Start -> y if(nd()}{ return y0;} \\ Start -> +(Start,Start) if(nd()){ left = fStart(x0,y0); right = fStart(x0,y0); return left + right;} }

o1=fStart(x1,y1); o1 is 𝑔 𝐻 x1,y1 for some 𝑔 𝐻 in 𝑀 𝐻 o0=fStart(x0,y0); o0 is 𝑔 𝐻 x0,y0 for some 𝑔 𝐻 in 𝑀 𝐻 Can be different

𝐻 is non-deterministically drawn from 𝑀(𝐻) 𝑔 𝑝 ← 𝑔 Ԧ 𝐻 ( Ԧ 𝑦) (o0,o1) = Start(x0,y0); <int,int> fStart(x0,y0,x1,y1){ if(nd()){ return (0,0);} \\ Start -> 0 if(nd()){ return (1,1);} \\ Start -> 1 if(nd()){ return (x0,x1);} \\ Start -> x if(nd()}{ return (y0,y1);} \\ Start -> y if(nd()){ \\ Start -> +(Start,Start) (a0,a1) = Start(x0,y0,x1,y1); (b0,b1) = Start(x0,y0,x1,y1); return (a0+b0,a1+b1);} }

𝑡𝑧 𝐹 assert cannot be falsified unrealizable 𝑡𝑧 unrealizable

Evaluation

The tool NOPE Nope 𝑆𝑓 𝐹 Unreachable Seahorn Reduction UNREALIZABLE ESolver 𝑇𝑧 𝐹 Example Set 𝑄 𝐹 UNSAT SAT 𝑄 is a solution Veifier Z3 new example e

Application max 𝑦, 𝑧 = 𝐽𝑈𝐹(> (𝑦, 𝑧), 𝑦, 𝑧) Optimal ? QSyGUS [cav18] SyGuS QSyGuS Solution Solution Weighted 𝑋 cost 𝐷 1 Cost 1 ⋯ Specification 𝜚 𝜚, 𝐻 <𝑑 1 𝜚, 𝐻 𝜚, 𝐻 <1 Minimize 𝑑𝑝𝑡𝑢 𝑁𝑗𝑜𝑗𝑛𝑗𝑨𝑓 # 𝐽𝑈𝐹 (𝜒, 𝐻 <1 ) is unrealizable

Benchmarks 132 SyGuS benchmarks 60 SyGuS benchmarks QSyGuS which should be unrealizable

Overall performance of NOPE 132 variants of benchmarks taken from SyGuS Solved 1. bounded number of if-operators 13/57 2. bounded number of plus-operators 1/30 3. restricted range of constants 45/45 59/132

Limitation 1 of NOPE: number of examples

Limitation 2 of NOPE: size of grammars Large sized reachability problem

Open questions: Conclusion 1. reachability problem with large number of functions 2. beyond SyGuS Nope 𝑆𝑓 𝐹 Unreachable Seahorn Reudction UNREALIZABLE ESolver 𝑇𝑧 𝐹 Example Set 𝑄 𝐹 UNSAT SAT 𝑄 is a solution Verifier Z3 new example e

CEGIS may not Terminate 𝜒 𝑔(𝑦), 𝑦 = 𝑔 𝑦 > 𝑦 Start → +(Start, Start) 0 1 𝑔 𝑦 = max E + 1 Example Set 𝐹

Non Single-invocation Specification