Learning to Prove Theorems via Interacting with Proof Assistants - - PowerPoint PPT Presentation

Learning to Prove Theorems via Interacting with Proof Assistants

Kaiyu Yang, Jia Deng

Assumptions Conclusion

𝑜 ∈ ℕ 1 + 2 + ⋯ + 𝑜 = 𝑜 + 1 𝑜 2 ⇒

Automated Theorem Proving (ATP)

Automated theorem prover Assumptions Conclusion

𝑜 ∈ ℕ 1 + 2 + ⋯ + 𝑜 = 𝑜 + 1 𝑜 2 ⇒

Automated Theorem Proving (ATP)

Automated theorem prover Proof Assumptions Conclusion

𝑜 ∈ ℕ 1 + 2 + ⋯ + 𝑜 = 𝑜 + 1 𝑜 2 ⇒

Automated Theorem Proving (ATP)

Computer-aided proofs in math

Automated Theorem Proving (ATP) is Useful for

Computer-aided proofs in math Software verification

Automated Theorem Proving (ATP) is Useful for

Computer-aided proofs in math Hardware design Software verification

Automated Theorem Proving (ATP) is Useful for

Computer-aided proofs in math Hardware design Software verification Cyber-physical systems

Automated Theorem Proving (ATP) is Useful for

Drawbacks of State-of-the-art ATP

Conjunctive normal forms (CNFs)

1 + 2 + ⋯ + 𝑜 = 𝑜 + 1 𝑜 2 𝑞⋁¬𝑟⋁¬𝑠⋁𝑡 ¬𝑦⋁𝑧⋁𝑨⋁𝑟

Theorem

• Prove by resolution

Drawbacks of State-of-the-art ATP

Conjunctive normal forms (CNFs)

1 + 2 + ⋯ + 𝑜 = 𝑜 + 1 𝑜 2 𝑞⋁¬𝑟⋁¬𝑠⋁𝑡 ¬𝑦⋁𝑧⋁𝑨⋁𝑟 ¬𝑦⋁𝑧⋁𝑨⋁𝑞⋁¬𝑠⋁𝑡

Theorem

• Prove by resolution

Drawbacks of State-of-the-art ATP

Conjunctive normal forms (CNFs)

1 + 2 + ⋯ + 𝑜 = 𝑜 + 1 𝑜 2 𝑞⋁¬𝑟⋁¬𝑠⋁𝑡 ¬𝑦⋁𝑧⋁𝑨⋁𝑟 ¬𝑦⋁𝑧⋁𝑨⋁𝑞⋁¬𝑠⋁𝑡 …

Theorem

• Prove by resolution

• The CNF representation
• Long and incomprehensible even for simple math equations
• Unsuitable for human-like high-level reasoning

Drawbacks of State-of-the-art ATP

Conjunctive normal forms (CNFs)

1 + 2 + ⋯ + 𝑜 = 𝑜 + 1 𝑜 2 𝑞⋁¬𝑟⋁¬𝑠⋁𝑡 ¬𝑦⋁𝑧⋁𝑨⋁𝑟 ¬𝑦⋁𝑧⋁𝑨⋁𝑞⋁¬𝑠⋁𝑡 …

Theorem

Interactive Theorem Proving

Human Proof assistant

assumptions conclusion goal

Interactive Theorem Proving

Proof assistant Human

assumptions conclusion goal tactic

Interactive Theorem Proving

Proof assistant Human

Interactive Theorem Proving

Proof assistant Human

Interactive Theorem Proving

Proof assistant Human

Interactive Theorem Proving

Proof assistant Human

Interactive Theorem Proving

Proof assistant

Labor-intensive, requires extensive training

Human

Interactive Theorem Proving

Proof assistant Agent Human

• Tool for interacting with the Coq proof assistant
• 71K human-written proofs, 123 Coq projects
• Diverse domains
• math, software, hardware, etc.

[Barras et al. 1997]

CoqGym: Dataset and Learning Environment

• Tool for interacting with the Coq proof assistant
• 71K human-written proofs, 123 Coq projects
• Diverse domains
• math, software, hardware, etc.
• Structured data
• Proof trees
• Abstract syntax trees

Proof tree

[Barras et al. 1997]

CoqGym: Dataset and Learning Environment

Tactic

ASTactic: Tactic Generation with Deep Learning

𝑜, 𝑙 ∈ ℕ 𝑜 = 2𝑙 𝑜 ≥ 𝑙 Proof goal

ASTactic: Tactic Generation with Deep Learning

Proof goal Abstract syntax trees (ASTs) 𝑜, 𝑙 ∈ ℕ 𝑜 = 2𝑙 𝑜 ≥ 𝑙

Proof goal Abstract syntax trees (ASTs) 𝑜, 𝑙 ∈ ℕ 𝑜 = 2𝑙 𝑜 ≥ 𝑙

ASTactic: Tactic Generation with Deep Learning

Feature vectors

TreeLSTM encoder [Tai et al. 2015]

decoder Tactic AST

ASTactic can augment state-of-the-art ATP systems [Czajka and Kaliszyk, 2018] to prove more theorems

ASTactic: Tactic Generation with Deep Learning

Proof goal Abstract syntax trees (ASTs) 𝑜, 𝑙 ∈ ℕ 𝑜 = 2𝑙 𝑜 ≥ 𝑙 Feature vectors

TreeLSTM encoder [Tai et al. 2015]

• CoqHammer [Czajka and Kaliszyk, 2018]
• SEPIA [Gransden et al. 2015]
• TacticToe [Gauthier et al. 2018]
• FastSMT [Balunovic et al. 2018]
• GamePad [Huang et al. 2019]
• HOList [Bansal et al. 2019] (concurrent work at ICML19)

Related Work

Main differences:

• Our dataset is larger covers more diverse domains.
• Our model is more flexible, generating tactics in the form of ASTs.

Poster today @ Pacific Ballroom #247 Code: https://github.com/princeton-vl/CoqGym

Learning to Prove Theorems via Interacting with Proof Assistants

Kaiyu Yang, Jia Deng