Distributed Constraint Optimization DSA-1, MGM-1 (exchange - PowerPoint PPT Presentation

10/05/2014 Approximate Algorithms: outline • No guarantees Distributed Constraint Optimization – DSA-1, MGM-1 (exchange individual assignments) (Approximate approaches) – Max-Sum (exchange functions) • Off-Line guarantees – K-optimality and extensions • On-Line Guarantees – Bounded max-sum UAVs Cooperative Monitoring Why Approximate Algorithms • Motivations – Often optimality in practical applications is not achievable Video Streaming – Fast good enough solutions are all we can have • Example – Graph coloring Coordination – Medium size problem (about 20 nodes, three colors per node) – Number of states to visit for optimal solution in the worst Task Requests case 3^20 = 3 billions of states (Interest points ) • Key problem – Provides guarantees on solution quality Joint work with F. M. Delle Fave, A. Rogers, N.R. Jennings 1

10/05/2014 Task utility Example Prob. Task completion Priority UAV 2 T 2 T 1 UAV Urgency 3 UAV 1 First assigned UAVs reaches task T 3 Last assigned UAVs leaves task (consider battery life) Possible Solution DCOP Representation D  UAV x { T , T } 2 2 P=10,U=10,D=5 2 1 2 P=10,U=10,D=5 T F ( x , x ) 2 F ( x , x ) 1 1 2 T 2 2 3 B=5 1 UAV 3 UAV x x 1 1 P=10,U=10,D=5 3 F 3 x ( 3 ) B=5 T B=5 D  3 D  { 1 T } { , } T T 1 3 2 3 2

10/05/2014 Types of Guarantees Centralized Local Greedy approaches • Greedy local search Instance-specific Accuracy: high alpha – Start from random solution Generality: less use of Bounded Max- instance specific knowledge – Do local changes if global solution improves Sum – Local: change the value of a subset of variables, usually one DaCSA -1 -1 -1 Accuracy -4 Instance-generic -1 0 No guarantees K-optimality 0 -2 MGM-1, -1 -1 T-optimality -2 DSA-1, Region Opt. Max-Sum Generality 0 Centralized Local Greedy approaches Distributed Local Greedy approaches • Local knowledge • Problems • Parallel execution: – Local minima – A greedy local move might be harmful/useless – Standard solutions: RandomWalk, Simulated Annealing – Need coordination -1 -1 -1 -1 -1 -2 -4 -1 -1 -1 0 -2 0 -2 0 0 -2 -2 -1 -1 -1 -1 -1 -1 -4 3

10/05/2014 Distributed Stochastic Algorithm DSA-1: Execution Example • Greedy local search with activation probability to mitigate issues with parallel executions rnd > ¼ ? rnd > ¼ ? rnd > ¼ ? rnd > ¼ ? • DSA-1: change value of one variable at time -1 -1 -1 • Initialize agents with a random assignment and P = 1/4 -1 communicate values to neighbors • Each agent: 0 -2 – Generates a random number and execute only if rnd less than activation probability – When executing changes value maximizing local gain – Communicate possible variable change to neighbors DSA-1: discussion Maximum Gain Message (MGM-1) • Extremely “cheap” (computation/communication) • Coordinate to decide who is going to move • Good performance in various domains – Compute and exchange possible gains – Agent with maximum (positive) gain executes – e.g. target tracking [Fitzpatrick Meertens 03, Zhang et al. 03], • Analysis [Maheswaran et al. 04] – Shows an anytime property (not guaranteed) – Benchmarking technique for coordination – Empirically, similar to DSA – More communication (but still linear) • Problems – No Threshold to set – Activation probability must be tuned [Zhang et al. 03] – Guaranteed to be monotonic (Anytime behavior) – No general rule, hard to characterise results across domains 4

10/05/2014 MGM-1: Example Local greedy approaches • Exchange local values for variables – Similar to search based methods (e.g. ADOPT) -1 -1 • Consider only local information when maximizing – Values of neighbors • Anytime behaviors 0 -1 -1 -2 • Could result in very bad solutions -1 -1 0 -2 G = -2 G = 0 G = 2 G = 0 Max-sum Max-Sum on acyclic graphs Agents iteratively computes local functions that depend X1 only on the variable they control • Max-sum Optimal on acyclic graphs Util Value – Different branches are independent – Each agent can build a correct X1 X2 X2 Choose arg max estimation of its contribution to the global problem (z functions) – We can use DPOP: Util and Value X4 X3 Shared constraint – Separator size always 1  X3 X4 polynomial computation/comm. All incoming messages except x2 All incoming messages 5

10/05/2014 Max-Sum on cyclic graphs Util Message Schedules • Max-sum on cyclic graphs • New messages replace old ones X1 – Different branches are NOT • Messages are computed based on most recent messages independent • different possible schedules: Flooding, Serial – Agents can still build an (incorrect ) estimation of their contribution to X2 Flooding X1 Serial X1 the global problem – Propagate Util messages until convergence or for fixed amount of cycles X3 X4 X2 X2 – Each agent computes z-function and select the argmax. X3 X3 X4 X4 Max-Sum and the Factor Graph Constraint Graphs vs. Factor Graphs • Factor Graph x – [Kschischang, Frey, Loeliger 01] x F ( x , x ) r ( x ) 2 2 1 1 2   F x 2 – Computational framework to represent factored computation 2 2 x F ( x , x ) F ( x , x ) – Bipartite graph, Variable - Factor 1 1 2 2 2 3 1  q ( x )    x F 2 ( ) ( ) 2 1 F x F i x x 1 x r ( 1 x ) F ( x , x ) m ( x ) i 3   m ( 1 x )   F x 3 2 2 2 1 2   1 1 2 1 x UAV 3 F 3 x ( 3 ) x 2 F ( x , x ) F 3 x ( ) 2 1 1 2 ( , ) F x x 3 T 2 1 2 2 x T 1 1 UAV 1 UAV 3 F 3 x ( ) 3 T 3 x 3 6

10/05/2014 Max-Sum on Factor Graphs Max-Sum on Factor Graphs x x F ( x , x ) F ( x , x ) 2 2 1 1 2 1 1 2 r ( x )   F x 2 2 2 x x 1 1 q ( x ) q ( x )     r ( 1 x ) x F 2 x F 2 2 1   2 3 F x 3 1 r ( x )   F x 2 3 2 F ( x , x ) F ( x , x ) q ( 3 x ) 2 1 2   2 1 2 x F 3 3 F ( x , x , x ) F ( x , x , x ) x x 3 1 2 3 3 1 2 3 3 3 sum up info from other nodes local maximization step Constraint Graphs vs. Factor Graphs (Loopy) Max-sum Performance • Good performance on loopy networks [Farinelli et al. 08] UAV 2 – When it converges very good results T • Interesting results when only one cycle [Weiss 00] 2 T 1 – We could remove cycle but pay an exponential price (see DPOP) UAV 3 – Java Library for max-sum http://code.google.com/p/jmaxsum/ T 3 F ( x , x ) x F ( x , x ) 1 1 2 2 1 1 2 F ( x , x ) F ( x , x ) x 2 1 2 2 2 3 x 2 1 F ( x , x x ) 3 1 2 , 3 x 1 x 3 x F ( x , x , x ) 3 3 1 2 3 7

10/05/2014 Max-sum on hardware Max-Sum for low power devices • Low overhead – Msgs number/size • Asynchronous computation – Agents take decisions whenever new messages arrive • Robust to message loss Quality guarantees for approx. UAVs Demo techniques • Key area of research • Address trade-off between guarantees and computational effort • Particularly important for many real world applications – Critical (e.g. Search and rescue) – Constrained resource (e.g. Embedded devices) – Dynamic settings 8

10/05/2014 Instance-generic guarantees Guarantees on solution quality • Key Concept: bound the optimal solution Instance-specific – Assume a maximization problem Bounded Max- Characterise solution quality without Sum – optimal solution, a solution running the algorithm – DaCSA Accuracy – percentage of optimality Instance-generic • [0,1] No guarantees • The higher the better K-optimality – approximation ratio MGM-1, T-optimality DSA-1, • >= 1 Region Opt. Max-Sum • The lower the better – is the bound Generality K-Optimality framework K-Optimal solutions • Given a characterization of solution gives bound on solution quality [Pearce and Tambe 07] 1 • Characterization of solution: k-optimal 1 1 1 1 1 • K-optimal solution: 1 – Corresponding value of the objective function can not be 1 improved by changing the assignment of k or less 2-optimal ? Yes 3-optimal ? No variables. 2 2 0 0 1 0 0 2 9

10/05/2014 Bounds for K-Optimality K-Optimality Discussion • Need algorithms for computing k-optimal solutions For any DCOP with non-negative rewards [Pearce and Tambe 07] – DSA-1, MGM-1 k=1; DSA-2, MGM-2 k=2 [Maheswaran et al. 04] Maximum arity of constraints Number of agents – DALO for generic k (and t-optimality) [Kiekintveld et al. 10] • The higher k the more complex the computation (exponential) Percentage of Optimal: K-optimal solution • The higher k the better Binary Network (m=2): • The higher the number of agents the worst Trade-off between generality and Trade-off between generality and solution quality solution quality • K-optimality based on worst case analysis • Knowledge on reward [Bowring et al. 08] • assuming more knowledge gives much better bounds • Beta: ratio of least minimum reward to the maximum • Knowledge on structure [Pearce and Tambe 07] 10