Reducing the Search Space of Resource Constrained DCOPs T. Matsui 1) - PowerPoint PPT Presentation

Reducing the Search Space of Resource Constrained DCOPs T. Matsui 1) , M. Silaghi 2) , K. Hirayama 3) , M. Yokoo 4) , B. Faltings 5) and H. Matsuo 1) 1) Nagoya Institute of Technology, 2) Florida Institute of Technology, 3) Kobe University, 4) Kyushu University, 5) Swiss Federal Institute of Technology, Lausanne (EPFL) 1

Contribution  Resource Constrained Distributed Constraint Optimization Problem (RCDCOP) is a dedicated representation for multi-agent cooperation constrained by shared resources.  A Dynamic programming method based on pseudo-trees can be applied to the RCDCOP.  Its large search space increases the size of tables of the dynamic programming method.  We apply combinations of several efficient methods to reduce the search space for the RCDCOP solver. 2

Outline  RCDCOP  Problem definition  Pseud-trees ignoring Resource Constraints  A solver based on dynamic programming  Reducing search space  Reducing redundancy  Improving accuracy of infeasibility/feasibility  Experiments  Conclusions 3

Distributed Constraint Optimization Problem (DCOP) Multi-agent cooperation is formalized as constraint optimization . State/decision of an agent is represented as a variable . Relationship between agents is represented as a constraint . x : variable/agent Variable i (Agent) x D : domain of i i c , x 0 Constraint i j c , (objective function) x 1 x x : constraint between and i j i j x 4 c , f , : objective function for i j x 2 i j x 3  f , min : global optimal cost i j for all constraint s Constraint network for DCOP Distributed search algorithms solve the global optimal solution . 4

Resource Constrained DCOP (RCDCOP) [Matsui et al. 08] In RCDCOP , Resource Constraint defines limitations of resource use. A resource constraint is a global (n-ary) constraint. r : resource. a Resource A resource has its capacity. constraint R r : capacity of r 1 a a r 0 x 0 x 1 Each agent requires related resources u ( r , d ) : resource use of x 4 ( x i d , ) i i i x 2 x 3 Total use of a resource must not exceed its capacity . RDCCOP c : resource constraint s.t. a  u ( r , d ) R ≤ i a i a 5 i ∈ nodes which require r a

Pseudo-tree ignoring resource constraints [Matsui et al. 08] Several exact solvers for RCDCOPs employ extended Pseudo-trees that are based on spanning-trees (e.g. DFS trees) for the problems. Tree edge based on a DFS tree x 0 Back edge x 1 x 3 x 4 x 2 Additional hyper edge for r 1 resource constraint Constraints between sub-trees are: r 0 not arrowed original constraints arrowed resource constraints A resource constraint is just a Boolean constraint that depends on the summation of unary usage functions . The summation only depends on tree edges . The Boolean constraint is evaluated later using the summation. 6

Solver based on dynamic programming A dynamic programming DPOP (for DCOPs) employs pseudo trees. [Petcu et al. 05] [Kumar et al. 09] An extension of DPOP solves DCOPs with specific resources. We employ another DPOP for RCDCOPs (RC-DPOP) . The processing consists of two phases : VALUE COST(UTIL) x 0 message x 0 message x 1 x 3 x 4 x 1 x 3 x 4 x 2 x 2 r 1 r 1 r 0 r 0 1) computation of optimal cost 2) computation of optimal assignment Infeasible assignments are eliminated . In the following, we mainly focus on the computation of the optimal cost . 7

Computation of cost values Each agent i computes/propagates a table of:  assignments for ancestor variables that relate with sub-tree rooted at i  cost values g i for the assignments  combination of resource use that are represented by virtual variables vr l vr 1 '' x 0 x 1 g 3 x 0 g 1 vr 0 vr 1 x 0 0 5 0 0 1 0 3 0 x 1 0 1 2 4 x 2 x 3 vr 0 ' vr 1 ' x 1 g 2 r 0 r 1 For the same assignments of variables, different combinations of cost value and resource use can exist. (The cost values implicitly depend on the resource use.) 8

Computation of cost values (contd.) The aggregation of cost values is based on: 1) summation of cost values for each combination of assignments and resource use 2) minimization for all values of current variable Infeasible assignments are eliminated . minimization u ( r 0 , x 1 ) + vr 0 ' vr 1 0 1 0 vr 0 vr 1 ' + vr 1 '' 0 0 4 x 0 0 0 1 x 1 x 0 g 1 vr 0 vr 1 x 0 0 5 4 6 f 0,1 f 0,1 + g 2 + g 3 1 9 3 3 2 7 8 2 x 1 u ( r 0 , x 1 ) summation x 2 x 3 vr 1 '' x 0 x 1 g 3 vr 0 ' vr 1 ' x 1 g 2 r 0 r 1 The size of tables grows with the number of combinations of resource use. 9

Reducing the search space To reduce the size of the tables in DP, we employ preprocessing and add-on processing based on:  Root node for each resource 1)  Aggregation of cost value and resource use 1)  Global lower bound of resource use 1), 2)  Replacing evaluations of resource use 2)  Aggregation of feasible assignments 1), 3) The methods 1) reduce redundancy or 2) improve accuracy of infeasibility/ 3) feasibility. Several ideas are based on previous studies. We mainly investigate effects of the methods for RCDPOP. 10

Root node for each resource For resource r , the highest agent that relates to r eliminates resource use vr of r . ε ε g 1 vr 0 , vr 1 g 1 vr 1 root agent of r 1 root agent of r 0 , r 1 (preprocessing) root agent of r 0 x 0 g 1 vr 0 , vr 1 ( eliminates vr 0 ) x 0 x 0 x 0 g 1 vr 0 , vr 1 x 1 x 1 vr 0 , vr 1 x 1 g 2 x 2 x 3 x 2 x 3 vr 0 , vr 1 x 1 g 2 r 0 r 0 r 1 r 1 The resource use is propagated among limited area of agents. 11

Aggregation of cost value and resource use For the same assignments , if combination w' of cost value and resource use are covered by another combination w , combination w' is replaced by w . If combination w' has higher cost value and resource use than other combinations, combination w' is never preferred. cost value and resource use x 0 g 1 vr 0 , vr 1 a b' c' d' w' w' w' a b c d w w w x 1 g 1 v r 0 v r 1 g 1 v r 0 v r 1 (a) w' is replaced by w (b) w and w' are not aggregated Redundant assignments in the table are eliminated . 12

Global lower bounds of resource use Each lower bound of resource use is subtracted/propagated to the root. Capacity of the resource is reduced by the global lower bound . Global lower bound Resource use function u for r 0 0 -3 Reduced capacity cr 0 of r 0 x 0 u 0 ( r 0 ,x 0 ) x 0 u 0 ( r 0 ,x 0 ) -1 is propagated. 1 0 0 0 0 0 (preprocessing) 1 0 1 1 cr 0 ← cr 0 - 3 2 2 1 2 (preprocessing) x 0 x 0 x 1 u 1 ( r 0 ,x 1 ) x 1 u 1 ( r 0 ,x 1 ) -1 1 0 x 0 0 0 0 0 x 1 x 1 2 1 1 1 2 1 2 2 x 1 x 2 x 2 r 0 x 2 u 2 ( r 0 ,x 2 ) x 2 u 2 ( r 0 ,x 2 ) x 2 -1 1 0 0 0 0 0 Accuracy of infeasibility 2 1 1 1 13 3 2 2 2 is improved.

Replacing evaluations of resource use Each resource use function is decomposed into agents that are: 1) constrained with the agent that originally has the resource use and 2) one of the lowest agent for each path of the pseudo-tree. u 0 ( r 0 ,x 0 ) x 0 u 0 ( r 0 ,x 0 ) x 0 a a 0 x 0 0 0 0 x 0 (preprocessing) b 2 b 0 c 5 c 0 x 1 x 3 x 1 x 3 u 0 ,3 ( r 0 ,x 0 ) x 0 x 2 u 0 ,2 ( r 0 ,x 0 ) x 0 x 2 a 0 0 a 0 0 b 1 b 1 r 0 c 3 r 0 c 2 (a) original problem (b) equivalent problem Accuracy of infeasibility is improved in the lower agents . 14

Aggregation of feasible assignments If the global upper bound of resource use does not exceed the capacity, assignments are aggregated ignoring the resource use . x 0 x 0 cost and resource use x 1 x 1 w' cr 0 w' w w x 2 capacity of r 0 x 2 cost v r 0 cost v r 0 r 0 r 0 In both cases, w' is replaced by w . upper bound (maximum use) exact resource use ( Only cost values are compared .) for other parts for current sub-tree (preprocessing) The aggregation complementally effects with the pruning based on infeasibility. 15

Experiments Efficiency of the proposed methods are evaluated using example RCDCOPs:  n ternary variables  1.1 n binary constraints (low density)  r n-ary resource constraints  each resource is shared by n / r agents  range of cost values is {1, …, 5}  range of the resource use for each variable is {1, …, u }  total amount of the resource is c ・ u ・ ( n / r ) ( c : parameter) Combinations of the proposed methods are compared. The number of assignments in tables is evaluated. 16

Effects of additional methods vi: base line (#variables: n=20, max. res. use: u=5, capacity of res.: c=0.25) vir: + trimming root nodes 10000 vira: + aggregation of assign. total viras: + global lower bounds max. virasd: + replacing evaluation 1000 virasdf: + feasibility num. of assignments 100 10 1 vi virasdf vi virasdf vi virasdf vir vira viras virasd vir vira viras virasd vir vira viras virasd r=1 r=4 r=10 (#resources) While most methods reduces the number of assignments , using “feasibility” has no effect because of low feasibility ( c=0.25 ). 17

Effects of feasibility virasd: without feasibility (total) (#variables: n=20, max. res. use: u=5) (max.) 10000 virasdf: +feasibility (total) (max.) 1000 num. of assignments 100 10 1 min.0.25 0.5 0.75 1 min.0.25 0.5 0.75 1 min.0.25 0.5 0.75 1 infeasible feasible r=1 r=4 r=10 (#resources) ratio c for capacity of resources In the case of relatively feasible problems , 18 using “feasibility” reduces the number of assignments.