Guarantees in in Program Synthesis Qinheping Hu , Jason Breck , John Cyphert , Loris D'Antoni , Thomas Reps
Program Synthesis Specification Solution Program Synthesizer Search space Program Synthesis is Un Unpredicta edictable ble
Program Synthesis is Un Unpredictable edictable Search space Solution space
Program Synthesis is Un Unpredictable edictable Specification Solution Program Synthesizer Search space Ability to prefer a solution when there are multiple solution
Program Synthesis is Un Unpredictable edictable Specification: π π¦ = π¦ Solution space Search space constant programs: 1,2,3,β¦
Program Synthesis is Un Unpredictable edictable Solution Program ? Search-based synthesizer timeout Solution space Search space
Gu Guarantees ntees in Program Synthesis Specification Solution Program Synthesizer Search space Ability to answer no solutions Ability to prefer a solution when there are multiple solution Make program synthesis predictable
Syntax-Guided Synthesis with Quantitative Objectives [CAV18]
Syntax-Guided Synthesis (SyGuS) π πππ¦(π¦, π§), π¦, π§ : πππ¦ π¦, π§ β₯ π¦ β§ πππ¦ π¦, π§ β₯ π§ β§ (πππ¦ π¦, π§ = π¦ β¨ πππ¦ π¦, π§ = π§) Specification Solution Program Synthesizer Search space π β π(π») such that βπ¦, π§. π π, π¦, π§ Start = +(Start, Start) | π½ππΉ(BExpr, Start, Start) π¦ π§ 0 1 max π¦, π§ = π½ππΉ(> (π¦, π§), π¦, π½ππΉ(< π¦, π§ , π§, π¦) BExpr = πππ’(BExpr ) | > (Start, Start) |π΅ππ(BExpr, BExpr)
A limitation of SyGuS What we expected Outp tput of f th the CVC4 solv lver
Features you want Optim imization Obje jectives Readable Smallest size Efficient Least number of multiplication Most likely Largest probability
Gu Guarantees ntees in SyGuS Specification Solution Program SyGuS solver Search space Quantitative objectives
Adding Quantitative Objective to SyGuS Start = Start + Start BExpr = Start > Start | ππ BExpr π’βππ Start πππ‘π Start | πππ’ BExpr x π§ 0 1 | BExpr πππ BExpr
Adding Quantitative Objective to SyGuS Start = Start + Start BExpr = Start > Start | ππ BExpr π’βππ Start πππ‘π Start | πππ’ BExpr x π§ 0 1 | BExpr πππ BExpr Quantitative objective: Minimize number of if-statement
Adding Quantitative Objective to SyGuS Start = Start + Start /0 BExpr = Start > Start /0 | ππ BExpr π’βππ Start πππ‘π Start /1 | πππ’ BExpr /0 x/0 π§/0 0/0 1/0 | BExpr πππ BExpr /0 Quantitative objective: Minimize number of if-statement
ITE:1 If(x>y) ITE:1 >:0 Y:0 then = if(x > x) then 0 else x Else y X:0 Y:0 X:0 >:0 0:0 X:0 X:0 Weight = 2
Adding Quantitative Objective to SyGuS Weighted grammar Start = Start + Start /0 BExpr = Start > Start /0 | ππ BExpr π’βππ Start πππ‘π Start /1 | πππ’ BExpr /0 x/0 π§/0 0/0 1/0 | BExpr πππ BExpr /0 Quantitative objective: Minimize number of if-statement
Adding Quantitative Objective to SyGuS Weighted grammar Start = Start + Start /0 BExpr = Start > Start /0 | ππ BExpr π’βππ Start πππ‘π Start /1 | πππ’ BExpr /0 x/0 π§/0 0/0 1/0 | BExpr πππ BExpr /0 Quantitative objective: Minimize number of if-statement Minimize weight QSyGuS: (specification,weighted grammar, Quantitative objective)
Solving QSyGuS
QSyGuS WTG π Constraint π Minimize π₯πππβπ’
SyGuS QSyGuS ignore WTG π weight CFG π» Constraint π Constraint π Minimize π₯πππβπ’
SyGuS QSyGuS Solutionβs WTG π weight π 1 CFG π» CFG π» <π 1 Constraint π Constraint π Constraint π Minimize π₯πππβπ’
SyGuS QSyGuS Solutionβs WTG π weight π 2 CFG π» CFG π» <π 1 CFG π» <π 2 Constraint π Constraint π Constraint π Constraint π Minimize π₯πππβπ’
SyGuS QSyGuS Solutionβs WTG π weight π 2 CFG π» CFG π» <π 1 CFG π» <π 2 β― Constraint π Constraint π Constraint π Constraint π Minimize π₯πππβπ’
SyGuS QSyGuS Solutionβs WTG π weight 2 CFG π» CFG π» <2 Constraint π Constraint π Constraint π Minimize π₯πππβπ’ CFG π» <2 WTG π Program π has π₯πππβπ’ < 2 iff π» <2 accept π
WTG π CFG π» <2 Grammar Reduction Idea: keep tr Id track of f th the weight in in th the non-terminals Start = Start + Start /0 BExpr = Start > Start /0 | ππ BExpr π’βππ Start πππ‘π Start /1 | πππ’ BExpr /0 x/0 π§/0 0/0 1/0 | BExpr πππ BExpr /0 π₯πππβπ’ < 2
WTG π CFG π» <2 Grammar Reduction Id Idea: keep tr track of f th the weight in in th the non-terminals Start = Start + Start /0 BExpr = Start > Start /0 | ππ BExpr π’βππ Start πππ‘π Start /1 | πππ’ BExpr /0 x/0 π§/0 0/0 1/0 | BExpr πππ BExpr /0 π₯πππβπ’ < 2 (Start, < 2) = Start, 0 | (Start, 1)
WTG π CFG π» <2 Grammar Reduction Idea: keep tr Id track of f th the weight in in th the non-terminals Start = Start + Start /0 BExpr = Start > Start /0 | ππ BExpr π’βππ Start πππ‘π Start /1 | πππ’ BExpr /0 x/0 π§/0 0/0 1/0 | BExpr πππ BExpr /0 π₯πππβπ’ < 2 (Start, < 2) = Start, 0 | (Start, 1) Start, 1 = Start, 0 + Start, 1 | Start, 1 + (Start, 0) | ππ BExpr, 0 π’βππ (Start, 0) πππ‘π (Start, 0) Start, 0 = Start, 0 + Start, 0 π¦ π§ 0 1 β―
Handling complex weight constraints Tree grammars are closed under Boolean operations Minimization Minimization Linear search 3 < π₯πππβπ’ Complement of π» <4 π» <5 β© π» >2 2 < π₯πππβπ’ < 5 π» π₯πππβπ’ 1 >3 β© π» π₯πππβπ’ 2 <0.5 3 < π₯πππβπ’ 1 and π₯πππβπ’ 2 < 0.5
Evaluation
Evaluation SyGuS QSyGuS WTG π CFG π» CFG π» <π 1 CFG π» <π 2 β― Constraint π Constraint π Constraint π Constraint π Minimize π₯πππβπ’ SyGuS solvers ESolver CVC4
Evaluation 26 Benchmarks taken from SyGuS 1.minimize number of specified operator, minimize solution size 2.maximize solution probability 3.find sorted optimal for (# of specified operators, size) 4.find Pareto optimal for (# of specified operators, size) Find solution with better cost for 16/26 SyGuS benchmarks Find optimal in 14/26 (couldnβt prove optimality for 2 benchmarks) Average time 3.1x Compared to SyGuS
Conclusion SyGuS QSyGuS Solutionβs WTG π weight π· 2 CFG π» CFG π» <π 1 CFG π» <π 2 β― Constraint π Constraint π Constraint π Constraint π Minimize π₯πππβπ’ optimal unrealizable
Gu Guarantees ntees in SyGuS Specification Solution Program SyGuS solver Search space Ability to answer no solutions Answering unrealizable Ability to prefer a solution when there are multiple solution Quantitative objectives
Tue 12:10 come to my CAV talk Proving Unrealizability for Syntax-Guided Synthesis [CAV19]
A Syntax-Guided Synthesis (SyGuS) is Specification Search space G: π π(π¦, π§), π¦, π§ : Start = +(Start, Start) π π¦, π§ β₯ π¦ β§ | π½ππΉ(BExpr, Start, Start) π π¦, π§ β₯ π§ β§ π¦ π§ 0 1 (π π¦, π§ = π¦ β¨ π π¦, π§ = π§) BExpr = πππ’(BExpr ) | > (Start, Start) |π΅ππ(BExpr, BExpr) Goal: find a program π β π(π») such that βπ¦, π§. π π, π¦, π§ max π¦, π§ = π½ππΉ(> (π¦, π§), π¦, π§)
Unrealizable SyGuS Problems Start = +(Start, Start) π¦ π§ 0 1 βπ¦, π§. max π¦, π§ β₯ π¦ β§ max π¦, π§ β₯ π§ β§ (max π¦, π§ = π¦ β¨ max π¦, π§ = π§) No Solution
CEGIS-based Frameowrk Nope ESolver Example Set π πΉ UNSAT SAT π is a solution Verifier Z3 new example e
CEGIS-based Frameowrk Nope ππ πΉ Unreachable Seahorn Reduction UNREALIZABLE ESolver Example Set π πΉ UNSAT SAT π is a solution Verifier Z3 new example e
SyGuS is unrealizable β reachability problem ππ πΉ is unsatisfiable Reachability Verifier ππ πΉ Unreachable Seahorn Reduction UNREALIZABLE
Gu Guarantees ntees in Program Synthesis Specification Solution Program Synthesizer Search space Ability to answer no solutions Ability to prefer a solution Answering unrealizable when there are multiple solution 1. Tue 12:10 come to my talk More quantitative objectives 2. Beyond SyGuS 1.Semantic quantitative objectives 2.Resource bounded synthesis
Recommend
More recommend