23/10/2012 Outline Special Topics in AI: Intelligent Agents and Multi-Agent Systems • Introduction – DCOP for MAS – how to model problems in the DCOP framework Distributed Constraint Optimization • Solution Techniques for DCOPs (Exact approaches, DPOP) – Exact algorithms (DCSP, DCOP) • DPOP – Heuristics and approximate algorithms (without/with quality guarantees) Alessandro Farinelli • DSA, MGM, Max-Sum; k-optimality, bounded max-sum Working together Decentralised Coordination Coordination problem: • Decentralised coordination: Local decision with local Choose agent’s individual actions so to maximise information a system-wide objective • Why Decentralised coordination ? – In general no benefit for computation or solution quality – Robustness Task allocation: • avoid single point of failure individual actions: which fire to tackle – Scalability system-wide objective: minimise total • Not enough bandwidth to communicate/process all extinguish time information solution: a joint action – Leads to problem decomposition • Each agent cares only of local neighbours 1
23/10/2012 Reference Applications DCOPs for Decentralized Coordination Cooperative Exploration Incident Management Why DCOPs for decentralized coordination ? • Well defined problem – Clear mathematical formulation that captures most important aspects – Many solution techniques • Optimal: ABT, ADOPT, DPOP, ... Environment monitoring Energy management • Heuristics: DSA, MGM, Max-Sum, ... • Solution techniques can handle large problems – compared for example to sequential dec. Making (MDP, POMDP) Modeling Problems as DCOP Target Tracking • Surveillance • Meeting Scheduling • Why decentralize – Robustness to failure and message loss 2
23/10/2012 Target Tracking - DCOP Meeting Scheduling L, R L, R E • Variables -> Cameras T1 T3 Better in [18:00 – 19:00] • Domains -> Camera actions Window13:00 – 20:00 Duration 1h – look left, look right D • Constraints – Overlapping cameras T2 – Related to targets Window 15:00 – 18:00 L, R Duration 2h • Diabolik, Eva • Why decentralize • Maximise sum of – Privacy constraints Meeting Scheduling - DCOP Constraint Networks { } X = X ,..., X a set of variables (e.g. meetings) 1 n [13 – 20] [15 – 18] { } BC D = D ,..., D 19:00 16:00 a set of discrete variable domains (e.g. time slots) 1 n BL PL { } C = C ,..., C a set of constraints (e.g., equality, non overlap, ) 1 m [15 – 18] S i ⊆ X Scope of constraint C i 16:00 Hard constraints Soft constraint BS PS [15 – 18] No overlap (Hard) 16:00 Equals (Hard) [13 – 20] Preference (Soft) 19:00 3
23/10/2012 Graphical Representation Binary constraint networks Primal Graph Hypergraph • Each constraint is defined over two variables • Every constraint network can be mapped to a binary constraint network but Dual Graph – Addition of variables/constraints – Add complexity Constraint graph Objectives for constraint networks • Link between two variables if they share at least one • Constraint Satisfaction Problem ( CSP ) constraint (i.e., primal graph) – Objective: find an assignment for all the variables in the – In general, constraint graph ≠ constraint network network that satisfies all constraints • Constraint Optimization Problems ( COP ) – Objective: find an assignment for all the variables in the network that satisfies all constraints and optimizes a global objective function � � � � ) � ( * X = arg max F X i i � � X i Global function: an aggregation (i.e., sum) of local functions F i X ( ) i 4
23/10/2012 Distributed Constraint Reasoning Benchmarking problems In a decentralized • Motivations context: – Analysis of complexity and optimality is not enough • Agents control – Need to empirically evaluate algorithms on the same problem Variables • Graph coloring • Agents communicate – Simple to formalise very hard to solve to solve the problem – Well known parameters that influence complexity • Number of nodes, number of colors, density (number of link/number of nodes) – Many versions of the problem • CSP, MaxCSP, COP Graph Coloring Graph Coloring - MaxCSP • Network of nodes • Optimization Problem • Nodes can take on various colors • Natural extension of CSP • Adjacent nodes should not have the same color • Minimise number of conflicts – If it happens this is a conflict CSP 0 -1 -4 Yes No 5
23/10/2012 Weighted Graph Coloring - COP Distributed COP • We focus on optimization • Optimization Problem • DCOP = Constraint Network + Agents A = { A ,..., A } • Conflicts have a weight 1 k • Where each agent: • Maximise the sum of weights of violated constraints – Controls a subset of the variables (typically just one) – Is only aware of constraints that involve the variables it COP -2 -1 controls -1 -1 -1 – Communicate only with neighbours (constraint graph) BC -1 -1 -1 -3 PL BL -3 -3 BS -2 -2 -2 PS DCOP Solution techniques Performance measures • Exact approaches • Solution quality – Guarantee optimal solution – Exponential coordination overhead – Optimality not always achievable, – ADOPT, DPOP, OptAPO – Optimality Guarantees • Heuristics • Coordination Overdead – Low coordination overhead – Computation: computation effort (time complexity) – No guarantees on optimality – Communication: number and size of messages (network load) – DSA, MGM, Max-Sum • Desirable properties (hard to quantify) • Approximate approaches – Robustness to failures, parallelism, flexibility, privacy – Low coordination overhead maintenance, etc. – Optimality guarantees – Bounded max-sum, k-optimality 6
23/10/2012 Exact Approaches I Exact Approaches II • ADOPT (Search based) [Modi et al 05] • DPOP (Inference) [Petcu and Faltings 07] – Distributed branch and bound – Distributed Bucket Elimination – Partial order based on a DFS search (pseudotree) – Partial order based on a DFS search (pseudotree) – Asynchronous (high parallelism, flexible) – Linear number of messages – Number of messages exponential in number of agents – Exponential message size (in width of DFS search tree) – DFS-tree width tipically much less than number of agents • Small messages but exponentially many • Few messages but exponentially large Dynamic Programming Pseudotrees: basic concepts Optimization Protocol • Pseudotree arrangement of a graph G 1. DFS-tree building (special case of Pseudo tree) – A rooted tree with the same node as G Constraint graph � DFS-Tree – – Adjacent nodes in G falls in the same branch of the tree – Token passing • Nodes in different branches do not share direct coinstraints 2. Utility propagation – A DFS visit of a graph induces a Pseudotree • Not every pseudotree can be obtained with a DFS visit – Compile information to compute optimal value – Util messages from leaves to root 3. Value Propagation – Root chooses optimal value and propagate decision Value messages from root to leaves – 7
23/10/2012 Building a DFS tree Building a DFS-tree: example • Traverse the graph using a recursive procedure st = 0 X1 et = 6 • Each time we reach Xi from Xj we mark Xi as visited and X1 st = 2 st = 4 state that Xj is the father of Xi (and Xi is a children of Xj) 1 et = 2 et = 4 • When a node Xi has a visited neighbour that is not its X3 X4 X2 4 2 parent we state that Xj is a pseudo-parent of Xi (and Xi is a pseudochildren of Xj) 2 4 X2 • Can be done with a distributed procedure: X4 X3 st = 1 – Each node need only to communicate with neighbours et = 5 – Token passing to propagate information (e.g., visited nodes) token movement time++ = each time token moves st: first time node received the token et: last time node sent the token Pseudotrees and Separator Util Propagation Aim: build a value function so that root agent can make Separator optimal decision. Definition Dynamic programming: provide only key information Basic Property Operative Definition Ci: children of node i 8
23/10/2012 Util Propagation: messages Message Computation Functions � tables (variable are all discrete) Aggregation � join operator (relational algebra) Maximization � projection (keeping most valuable tuples) Join-sum operator Util Message propagation 9
23/10/2012 Value Propagation Value Computation Aim: inform all agents about decision from above so that they can choose best values for their variables Can reuse stored tables for computing util messages Value Propagation: messages DPOP analysis • Synchronous algorithm • Linear number of messages but exponentially large • Messages (and computation) is exponential in separator size • Separator size � graph induced width with DFS ordering 10
Recommend
More recommend
