We know (nearly) nothing! But can we learn? ? Stephan Schulz schulz@eprover.org
Driving the State of the Art Calculus Implementation Search Control 2
Driving the State of the Art How to do What inference system to use? inferences e ffi ciently? Calculus Implementation Search Control Where to search for proofs? 2
Evolution of Calculus 10000 8000 6000 "E 0.2 FOF/Calc" E 0.2 FOF 4000 2000 0 0 50 100 150 200 250 300 3
Evolution of Implementation 10000 8000 6000 "E 0.2 FOF" E 0.2 FOF Fast 4000 2000 0 0 50 100 150 200 250 300 4
Evolution of Implementation 10000 8000 6000 E 1.8 Best E 1.8 Slow 4000 2000 0 0 50 100 150 200 250 300 4
Evolution of Search Control/Clause Selection 10000 8000 E 0.2 Goals 6000 E 0.2 Larry E 0.2 FOF E 0.2 SC 4000 2000 0 0 50 100 150 200 250 300 5
Evolution of Search Control/Literal Selection 10000 8000 E 0.2 SmallestNegLit 6000 E 0.2 MaxLComplexAvoidPosPred E 0.2 FOF 4000 2000 0 0 50 100 150 200 250 300 6
Compare and Contrast 10000 10000 8000 8000 E 0.2 Goals 6000 "E 0.2 FOF/Calc" 6000 E 0.2 Larry E 0.2 FOF E 0.2 FOF E 0.2 SC 4000 4000 2000 2000 0 0 0 50 100 150 200 250 300 0 50 100 150 200 250 300 10000 10000 8000 8000 E 0.2 SmallestNegLit 6000 6000 "E 0.2 FOF" E 0.2 MaxLComplexAvoidPosPred E 0.2 FOF Fast E 0.2 FOF 4000 4000 2000 2000 0 0 0 50 100 150 200 250 300 0 50 100 150 200 250 300 7
Improving heuristics has been the main source of progress in proof search! 8
. . . and our heuristics still suck! 1.0 Evolved 10SC11/FIFO SC11 0.8 FIFO 0.6 ratio 0.4 0.2 0.0 0 500 1000 1500 problems 9
Humans are Inadequate! ◮ We are not good at keeping large amounts of data in our head ◮ We are not good at analysing large amounts of data without help ◮ We are not good visualising complex relationships Compare “The Magical Number Seven, Plus or Minus Two” 10
Player of Games ◮ Chess State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where ◮ ◮ (Opening) Success: Capture of the king ◮ 11
Player of Games ◮ Chess State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where ◮ ◮ (Opening) Success: Capture of the king ◮ ◮ Go State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board ◮ 11
Player of Games ◮ Chess State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where ◮ ◮ (Opening) Success: Capture of the king ◮ ◮ Go State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board ◮ ◮ Saturating theorem proving P State: Set of clauses ◮ (processed clauses) Choice point: Which clause to process next? g = ☐ ◮ ? Simpli- Gene- fiable? rate ◮ Pick term ordering, literal selection strategy g Cheap Simplify Simplify Success: Derivation of the empty clause U ◮ (unprocessed clauses) 11
Player of Games ◮ Chess State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where ◮ ◮ (Opening) Success: Capture of the king ◮ ◮ Go State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board ◮ ◮ Saturating theorem proving P State: Set of clauses ◮ (processed clauses) Choice point: Which clause to process next? g = ☐ ◮ ? Simpli- Gene- fiable? rate ◮ Pick term ordering, literal selection strategy g Cheap Simplify Simplify Success: Derivation of the empty clause U ◮ (unprocessed clauses) 11
Player of Games ◮ Chess State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where ◮ ◮ (Opening) Success: Capture of the king ◮ ◮ Go State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board ◮ ◮ Saturating theorem proving P State: Set of clauses ◮ (processed clauses) Choice point: Which clause to process next? g = ☐ ◮ ? Simpli- Gene- fiable? rate ◮ Pick term ordering, literal selection strategy g Cheap Simplify Simplify Success: Derivation of the empty clause U ◮ (unprocessed clauses) 11
Player of Games ◮ Chess State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where ◮ ◮ (Opening) Success: Capture of the king ◮ ◮ Go State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board ◮ ◮ Saturating theorem proving P State: Set of clauses ◮ (processed clauses) Choice point: Which clause to process next? g = ☐ ◮ ? Simpli- Gene- fiable? rate ◮ Pick term ordering, literal selection strategy g Cheap Simplify Simplify Success: Derivation of the empty clause U ◮ (unprocessed clauses) 11
Player of Games ◮ Chess State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where ◮ ◮ (Opening) Success: Capture of the king ◮ ◮ Go State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board ◮ ◮ Saturating theorem proving P State: Set of clauses ◮ (processed clauses) Choice point: Which clause to process next? g = ☐ ◮ ? Simpli- Gene- fiable? rate ◮ Pick term ordering, literal selection strategy g Cheap Simplify Simplify Success: Derivation of the empty clause U ◮ (unprocessed clauses) 11
Player of Games ◮ Chess State: Different pieces on an 8x8 board ◮ Choice point: Which piece moves where ◮ ◮ (Opening) Success: Capture of the king ◮ ◮ Go State: Configuration of stones on a 19x19 board ◮ Choice point: Where to place the next stone ◮ Success: Control of larger area of the board ◮ ◮ Saturating theorem proving ? State: Set of clauses ◮ Choice point: Which clause to process next? ◮ ◮ Pick term ordering, literal selection strategy Success: Derivation of the empty clause ◮ 11
Grand Challenge Integrate Machine Learning and Symbolic Reasoning 12
Discussion ◮ Should we target domain-specific or more general search control knowledge? ◮ Deep learning or hand-selected features - which is better for learning search control knowledge? ◮ What is a better source for learning: Meta-information (success/failure, time to success, . . . ), full proofs, or even full search protocols? 13
Discussion ◮ Should we target domain-specific or more general search control knowledge? ◮ Deep learning or hand-selected features - which is better for learning search control knowledge? ◮ What is a better source for learning: Meta-information (success/failure, time to success, . . . ), full proofs, or even full search protocols? Discuss away! 13
Recommend
More recommend
