Interactions sur le fonctionnement dans les systèmes multi-agents ouverts et hétérogènes Interactions about Actions in Open and Heterogenous Multi-Agent Systems Soutenance d'Habilitation à Diriger des Recherches Nicolas Sabouret Lundi 20 novembre 2009 1/47
Problematics Agents that can Symbolic AI reasoning understand w hat they are doing Introspection ● What they can do ● How and Why ● When Explainations ● etc in Open & Heterogeneous Multi-Agent Systems Real World situations (ex: Ambient Computing) 2/47
Problematics (cont.) Distributed System behaviour ← entities + interactions Need to combine functionalities Open Services can (dis)appear at runtime Loosely coupled → no a priori information about others Heterogeneous Inconsistent models for data & actions Agent interactions and Human-Agent interactions 3/47
Two problems Management of Service composition semantic heterogeneity Discovery & Composition Explicit goal Implicit goal Incompatible representations → dynamic → learning → dynamic semantic interpretation choregraphy interactions Introspection! Very simple problems often turn out very difficult to solve... 4/47
Outline Related work in… Service composition Semantic heterogeneity Reinforcement learning & interactions The VDL model Service composition Learning interactions Semantic heterogeneity Conclusion & future work 5/47
Related work Open & Heterogenous MAS Service Semantic Reinforcement learning composition Heterogeneity & Interactions Introspection 6/47
Service composition [Shehory, 99] Multi-Agent [Aknine, 02] coordination Coalition formation [Müller, 06] [Ermolayev, 03] → workflow description → goal description Negociation Choreography Service protocols [Peltz, 03] Composition [Paurobally, 05] → service orchestration & choreography Orchestration Planing [Moreau, 08] [Durfee, 01] → syntactic service orchestration → task-oriented [Wu, 03] Ontologies → H TaskNets of Services Service Oriented [OWL-S, 04] [Traversore, 04] Architectures [WSDL, 03] → planing on service ontology 7/47
Service composition Service description Tasks decomposition Reasoning Choreography (static) Orchestration Service composition 8/47
Service composition Existing work A priori task decomposition A priori known set of possible actions → static service choreography Open MAS → dynamic service choerography Discover tasks at runtime → instrospection → interaction model 9/47
Related work Open & Heterogenous MAS Service Semantic Reinforcement learning composition Heterogeneity & Interactions Introspection 10/47
Semantic Heterogeneity Ontology KR model engineering Thesaurus [WN, 98] Semantic Semantic Networks Ontologies [OWL, 04] Heterogeneity [Laera, 07] → MAS protocol for onto alignment Ontology alignment Structure-based [Valencia, 04] [Breitman, 05] [Morge, 07] Semantic [Ichise, 03] Reference negociation ontology [van Diggelen, 06] [Aleksowski, 06] (Anemone) Instance-based 11/47
Semantic Heterogeneity Reference ontology → concept anchoring Semantic negociation → dynamic alignment Open & loosely coupled MAS → impossible or incomplete alignments Dynamic understanding of concepts → Introspection → Interaction protocol 12/47
Related work Open & Heterogenous MAS Service Semantic Reinforcement learning composition Heterogeneity & Interactions Introspection 13/47
Learning & Interactions Interaction protocols for learning Data exchange → learning acceleration Open MAS ? → Learning interactions Learning when to interact [Melo & Veloso, 08] Learning what to interact [Kasai & al., 08] Open MAS 14/47
Learning & Interactions Multi-Agent Other agents Systems Memory Delegation POMDPs [Dutech, 03] MDPs Asynchronism SMDPs Reinforcement Learning Asynchronous & open → Memory → Introspection → Interaction protocols 15/47
Introspection & Interaction models for reasoning about actions in open & heterogeneous MAS 16/47
Introspection & MAS Interactions for... Service Semantic Learning composition Heterogeneity Interactions Agent & Interaction Model 17/47
Architecture Planing, learning, decision taking... Cognitive Layer Runtime control Observers Interaction control Agent Ontology Interaction model ● Query → softbody ● Request → events Interaction ● Others Anchoring Layer → questions about actions! Introspection Softbofy View Language Preconditions & effects Design Layer Capacities Language 18/47
The VDL model XML tree rewriting [Gurevich, 95] Data v(val) → softbody Ontology (Concepts x Relations) typeof and includes → IC(c) [Seco, 04] Other relations → p(R) 19/47
The VDL model (2) Actions Preconditions – effets Effects on data → v(newval) Message sending → <snd,perf(rcv,ct)> Preconditions Events: evt(x 1 (val 1 ),...,x n (val n )) Event patterns → evt(x 1 ,...x n ) Boolean preconditions → vars(p) Context Context-Structure Structure 20/47
Generative bottom-up Capacities = acceptable events → Precondition evaluation eval e (p,evt) → true iff p ∈ P s ∪ P c s is true under evt eval c (p) → true iff p ∈ P c is true VDL code introspection (using precondition and data structure) → generation of all syntactically possible events ∀ p ∈ P s ∪ P cs ,eval e p ,e = true → Set of possible events E ∀ p ∈ P c eval c p = true → Set of currently impossible events F ∃ p ∈ P cs ,eval e p ,e = false ∨∃ p ∈ P c eval c p = false np(e) = set of failed preconditions 21/47
VDL interaction model Sender: AID Specific performatives Receivers: AIDs Performative query, inform, unknown Content Conv-id request, agree Message-id impossible, assert-cannot assert-can, clarify, suggest FIPA-ACL based what-can query-contraint not-understood, error <snd,p(rcv,c)> snd rcv 22/47
Interaction model (cont.) Query & al. Request & al. Agt 1 Agt 2 Agt 1 Agt 2 query(v) request(e) ALT ALT inform(v=val) agree unknown(v) impossible(np(e)) assert-cannot(e) assert-can(e' ∈ F) Generalisation: clarify(E) query-constraint(X,C) → set of variables → inform({v i =val i }) 23/47
Introspection & MAS Interactions for... Service Semantic Learning composition Heterogeneity Interactions 24/47
Service choreography Yasmine Charif Initial request (2004-2007) Service discovery Dynamic choreography Final answer → initiator agent 2 1 3 4 25/47
Service choreography Initiator - participants Delegation to all participants request → assert-can ? query-constraint → query VDL → trigger sub-conversations Waiting for answers Convergence in EXPTIME → timeouts → Management of sub-conversations Message history <id,m 0 ,M,R> 26/47
[IAT, 07] Protocols init part query-constraint OPT query ALT inform part 1 part 2 unknown query n Agent k m n ALT OR k inform m unknown inform init part or unknown request part 1 part 2 OPT query history * assert-can ? answer to m 0 assert-can ALT query-constraint assert-can request OPT query assert-can agree inform 27/47
[RIA, 08] Example Implemented in Java on the VDL platform (2006) 28/47
Introspection & MAS Interactions for... Service Semantic Learning composition Heterogeneity Interactions 29/47
Learning interactions Shirley Hoet Goal → reward function (2008-?) Problems: Asynchronous → learn to wait Non-observable → POMDPs Delegation ( request ) → Memory action Internal action Send message Environment reward Wait Acquire requests Q-table Temperature Acquire queries Limited to… Lastest requests Memory or query-results 1 learning agent Performatives query & request 30/47
Learning interactions Acquiring requests Acquiring queries Agt 1 Agt 2 Agt 1 Agt 2 request what-can impossible suggest vars p , p ∈ NP E add(request) add(query) request query + timeouts + timeouts 31/47
Learning interactions Memory State + latest request(s) or query-result(s) [McCallum, 96] Iterative construction memory → Only some states are provided with a memory a1,a1 s1: a1>a2 a1 s1: a1>a2 a1,a2 s1: a1>a2 a2 a2 s1: a1>a2 s1: a2>a3 s1: a2>a3 s2: a3>a1>a4 s2: a3>a1>a4 s2: a3>a1>a4 a4 s4: a4>a1 s4: a4>a1 s4: a1>a3 etc 0 slot memory 1 slot memory 2 slot memory 32/47
[MFI, 09] Algorithm At each step rand W 〈 snd , what-can dest , ∅〉 evt answer ∈ ? A adjust W OR Q s, a / T e rand W prob a = ∑ b ∈ A e Q s ,a / T store answer send message query adjust Q(s,a) OR send message request OR store action perform action N cycles Add memory to k most ambiguous states: 3 rang s [ up s ] rang s [ − 1 [ q a 1 − q a 2 ] ∣ A s ∣ ∑ a ∈ A s q a ] rang s amb s = wait s 1 1 33/47
Recommend
More recommend
