A Formal Approach to Embedding First-Principles Planning in BDI - PowerPoint PPT Presentation

A Formal Approach to Embedding First-Principles Planning in BDI Agent Systems Mengwei Xu, Kim Bauters, Kevin McAreavey , Weiru Liu

Agent Frameworks Programming-Based Model-Based Belief-Desire-Intention (BDI) Hierarchical Task Networks (HTN) First-Principles Planning (FPP) Plan library Synthesise new plans Pros: designer intent, scalability Learning-Based Pros: robust, dynamic environments Cons: unforeseen situations Cons: expensive Reinforcement Learning

Extending BDI with FPP Programming-Based Model-Based Belief-Desire-Intention (BDI) Hierarchical Task Networks (HTN) BDI+FFP First-Principles Planning (FPP) BDI with FPP Extend existing BDI framework to account for incomplete Learning-Based plan library by synthesising new plans using FPP Some existing proposals but… Reinforcement Learning • Programmer must decide when to call FPP • Typically informal [Meneguzzi & de Silva, 2015]

BDI: Literature Software Platforms Jason [Bordini et al., 2007] Logics Jack [Winikoff, 2005] [Cohen & Levesque, 1990] Jadex [Pokahr et al., 2013] [Rao & Georgeff, 1991] [Shoham, 2009] Programming Languages Conceptual Agent Notation AgentSpeak [Rao, 1996] Extension of AgentSpeak that provides formal operational semantics CAN [Winikoff et al., 2002] CANPLAN [Sardina et al., 2011] Our framework = CAN-FPP Formal integration of CAN and FPP with operational semantics

CAN: Agent ℬ, Λ, Π Plan library Initial belief base Set of plan rules Belief base specifying agent’s initial beliefs Action description library Set of STRIPS-style action descriptions

CAN: Agent ℬ, Λ, Π Initial belief base Belief base specifying agent’s initial beliefs Belief base ℬ ⊆ ℒ Set of formulas from logical language ℒ ℬ must support: • ℬ ⊨ ( (Entailment) • ℬ ∪ ( (Addition) • ℬ ∖ ( (Deletion) Assume ℬ is a set of atoms

CAN: Agent ℬ, Λ, Π Action description library Set of STRIPS-style action descriptions Action description act ∶ ) ← ℬ + ; ℬ - Set of “add” atoms ℬ - ⊆ ℒ Primitive action symbol Set of “delete” atoms ℬ + ⊆ ℒ Precondition ) ∈ ℒ

CAN: Agent ℬ, Λ, Π Plan library Set of plan rules Context / ∈ ℒ Triggering event 4 Formula from ℒ e.g. belief update, new (sub)goal Plan rule 4 ∶ / ← % Body (program) % ∷= nil act ? / +1 − 1 ! 4 % 5 ; % % 5 ⊳ % % 5 ∥ % / 5 ∶ % 5 , … , / < ∶ % goal / ? , %, / @ 7 7 7 < Entailment Empty program Belief deletion Sequencing Concurrency CAN declarative goal Relevant plans Conditional Primitive action Belief addition New (sub)goal

Static entities that will be omitted CAN: Operational Semantics Basic configuration ℬ, Λ, Π, %, & Current intention Current belief base Remainder of a plan body to execute Belief base specifying agent’s current beliefs Execution history % = act , , act - , … , act / Sequence of primitive actions executed so far Current intention set Set of (partially executed) plan bodies Agent configuration ℬ, Λ, Π, %, Γ

CAN: Operational Semantics (Cont.) $ % $ & … $ ( Derivation Rule * Transition ! ⟶ !′ ) Transition between (basic or agent) configurations Specifies when/how an agent transitions to new configuration + / ℬ, -, +/ ⟶ ℬ ∪ / , -, nil Substitution 6= Substitution of variables in 6 with values from unifier = 4 ∶ 6 ← ℬ 8 ; ℬ : ∈ Λ 4= = act ℬ ⊨ 6= act ℬ ∖ ℬ 8 = ∪ ℬ : =, - D act, nil ℬ, -, act ⟶

Our Framework: CAN-FPP 1. Adds notion of “pure” declarative goals, in addition to “CAN” declarative goals • “Active” vs. “inactive” pure declarative goals • ”Procedural” vs. “declarative” intention sets 2. Adds instrinsic support for automatic calls to FPP Direct call (i.e. programmer specified) • Belief-driven call (via motivation library) • Recover-aid call (i.e. following plan failure) • 3. Extends operational semantics Adopting pure declarative goals • Planning for pure declarative goals • Managing (dropping, reactivating) pure declarative goals •

FPP Problem Λ, ℬ, $ % Goal Formula from $ % ∈ ℒ Action description library Set of STRIPS-style action descriptions Initial state Set of atoms ℬ ⊆ ℒ Offline solution sol ,-- Λ, ℬ, $ % = act 2 ; act 4 ; … ; act 6 Assumes classical planning Sequence of actions from Λ that is guaranteed to reach state ℬ′ from ℬ such that ℬ′ ⊨ $ % Could be extended to e.g. Online solution sol ,6 Λ, ℬ, $ % = act 2 conformant planning Next best action from Λ so as to reach state ℬ′ from ℬ′ such that ℬ ⊨ $ %

CAN-FPP: Agent ℬ, Π, Λ, ℳ Extended plan library Motivation library Set of extended plan rules Set of motivation rules [à la van Riemsdijk, 2004] Motivation rule / ⇝ goal / 0 , / 1 Extended plan rule 7 ∶ / ← &′ Triggering condition / ∈ ℒ Pure declarative goal Extended body (program) & ' ∷= & | goal / 0 , / 1 | activate goal / 0 , / 1 Pure declarative goal Reactivation program

CAN-FPP: Agent Configuration ℬ, Π, Λ, ℳ, &, Γ Motivation library ℳ is a static entity and will be omitted . ∪ Γ / Γ )* ∩ Γ = ∅ ,- ,- . ∪ Γ ,- / Extended intention set Γ = Γ )* ∪ Γ ,- . ∩ Γ / = ∅ Γ ,- ,- Procedural intention set Γ )* Standard CAN intention set . Active declarative intention set Γ ,- Set of “active” pure declarative goals / Inactive declarative intention set Γ ,- Set of “inactive” pure declarative goals . ∪ Γ / Declarative intention set Γ ,- = Γ ,- ,- Set of all pure declarative goals

CAN-FPP: Adopting Pure Declarative Goals ! ↑ = goal + , , + 0 ! ∈ Γ ! = goal + , , nil, + 0 Direct $% ? > 5678 5678 ℬ, 3, Γ Translate a CAN declarative goal to a pure < ∪ ! ↑ ℬ, 3, 4 $% ∖ ! , Γ declarative goal if it has no procedure :; + ⇝ ! ↑ ∈ ℳ ! ↑ = goal + , , + 0 ℬ ⊨ +C Belief-driven D > 5678 5678 ℬ, 3, Γ < ∪ ! ↑ C Trigger a motivation rule ℬ, 3, 4 :; ! ↑ = goal + , , + 0 ! ∈ Γ ! = goal + , , !′, + 0 ℬ ⊨ + 0 Recovery-aid $% F > 5678 5678 ℬ, 3, Γ Translate a CAN declarative goal to a pure < ∪ ! ↑ ℬ, 3, 4 $% ∖ ! , Γ declarative goal if the procedure is blocked :;

CAN-FPP: Planning for Pure Declarative Goals ! ↑ ∈ Γ ! ↑ = goal - . , - 0 sol <00 2, ℬ, - . = ! ' ! = act J ; … ; act = Offline planning %& ! ℱ HLL 7%8 ℬ, 5, Γ Generate plan for an active declarative goal and %& ∖ ! ↑ , Γ ℬ, 5, 6 FG ∪ goal - . , !, - 0 add plan to procedural intention set Online planning ! ↑ ∈ Γ ! ↑ = goal - . , - 0 sol <= 2, ℬ, - . = act ' ! = act ; activate ! ↑ %& Generate a single action for pure declarative ! ℱ HI 7%8 ℬ, 5, Γ goal, deactivate that goal, create plan to first D ∪ ! ↑ , Γ ℬ, 5, 6 FG ∪ goal - . , !, - 0 execute that action then to reactive the goal, %& and add plan to procedural intention set ! ↑ ∈ Γ ! ↑ = goal - . , - 0 ' sol 2, ℬ, - . = ⊥ Planning failure %& ! ℱ ; 7%8 ℬ, 5, Γ Drop pure declarative goal if planning fails %& ∖ ! ↑ ℬ, 5, 6 (alternatively: add goal to inactive declarative intention set)

CAN-FPP: Managing Pure Declarative Goals ! ↑ ∈ Γ ! ↑ = goal , - , , / ℬ ⊨ , - ∨ , / Drop pure declarative goal %& 9 %567 %567 ℬ, 3, Γ Drop declarative goal (whether active or inactive) if ℬ, 3, 4 %& ∖ ! ↑ success or fail condition is met ! ↑ ∈ Γ ! = activate ! ↑ ? ! ∈ Γ 75 %& Reactivate pure declarative goal E 5& @6AB ℬ, 3, Γ C ∪ ! ↑ Activate a pure declarative goal that is inactive ℬ, 3, 4 75 ∖ ! , Γ %&

CAN-FPP: Theoretical Results Offline FPP Goal state 9:; … 9:; ℬ = , 8 ⋅ !, nil ! ↑ = goal ( ) , ( + sol -++ /, ℬ, ( ) = ! ℬ, 8, ! ℬ′ ⊨ ( ) Theorem 1 (i) 9:; … 9:; ℬ = , 8 ⋅ !, nil ℬ, 8, ! ↑ Successful execution Offline FPP ! ↑ = goal ( ) , ( + sol -. /, ℬ, ( ) = act 3 ! = act 3 ; activate ! ↑ Goal state 9:; … 9:; ℬ = , 8 ⋅ act 3 ⋅ … ⋅ act . , nil ℬ, 8, ! ℬ′ ⊨ ( ) Theorem 1 (iii) 9:; … 9:; ℬ′, 8 ⋅ act 3 ⋅ … ⋅ act . , nil ℬ, 8, ! ↑ Successful execution

Future Work Mengwei Xu, Kim Bauters, Kevin McAreavey, and Weiru Liu. A framework for plan library evolution in BDI agent systems. In Proceedings of ICTAI'18 , to appear. Programming-Based Model-Based Hierarchical Task Networks (HTN) Belief-Desire-Intention (BDI) First-Principles Planning (FPP) CAN-FPP Learning-Based Reusable FPP plans Reinforcement Learning Expansion and contraction of plan library involving FPP plans

Questions? Thank you