Overcoming Limitations of Game-Theoretic Distributed Control Jason - PowerPoint PPT Presentation

Overcoming Limitations of Game-Theoretic Distributed Control Jason R. Marden California Institute of Technology (joint work with Adam Wierman) Southern California Network Economics and Game Theory Symposium October 1, 2009

Engineering systems Trend: Transition from centralized to local decision making range Network Coding Sensor coverage Vehicle Target Assignment Appeal Challenges Local processing (manageable) Characterization Reduces communication Coordination Robustness Efficiency How should we design distributed engineering systems?

Engineering systems Trend: Transition from centralized to local decision making range Network Coding Sensor coverage Vehicle Target Assignment Features of distributed design: Local decisions Game Theory Local information Global behavior depends on local decisions

Game theory social system Descriptive Agenda: Modeling game theory model as “game” Decision Global Makers Behavior Metrics: Reasonable description of sociocultural phenomena? Matches available experimental/observational data?

Game theory social system engineering system Prescriptive Agenda: Distributed robust optimization game theory model as desired “game” global behavior Decision Global Makers Behavior distributed control Metrics: Design parameters: Asymptotic global behavior? Decision makers Communication/Information requirement? Objective/Utility functions Computation requirement? Decision/Learning rule Convergence rates?

Big picture Game theory for distributed robust optimization Part #1: Part #2: model interactions as game local agent decision rules decision makers / players informational dependencies possible choices processing requirements local objective functions Goal: Emergent global behavior is desirable Appeal: Challenges: available distributed learning algorithms convergence rates? robustness to uncertainties self-interested users?

Big picture Game theory for distributed robust optimization Part #1: Part #2: model interactions as game local agent decision rules decision makers / players informational dependencies possible choices processing requirements local objective functions Goal: Emergent global behavior is desirable Appeal: Challenges: available distributed learning algorithms convergence rates? robustness to uncertainties self-interested users?

Outline Goal: Establish methodology for designing desirable utility functions Existence of (pure) NE Efficiency of NE Locality of information Tractability Budget balance Outline: - Propose framework to study utility design: Distributed welfare games - Identify methodologies that guarantees desirable properties - Identify fundamental limitations - Propose new framework to overcome limitations

Game theory Non-cooperative game: • Players: N = { 1 , 2 , ..., n } • Actions: a i ∈ A i • A = A 1 × ... × A n Joint actions: • Utilities: U i : A → R (preferences) U i ( a ) = U i ( a i , a − i ) (Pure) Nash equilibrium: U i ( a ∗ i , a ∗ a i ∈ A i U i ( a i , a ∗ − i ) = max − i )

Resource allocation games Setup: • Resources: R • Players: N • A i ⊆ 2 R Actions: W r : 2 N → R + • Welfare • � Global Welfare: W ( a ) = W r ( a r ) r player set that chose resource r Game design = Utility design

Resource allocation games Framework is common to many application domains range Network Coding Sensor coverage Vehicle Target Assignment Akella et al., 2002. (Congestion control) Goemans et al., 2004 (Content distribution) Kesselman et al., 2005. (Switching/congestion control) Komali and MacKenzie, 2007. (Topology control in ad-hoc networks) Campos-Nanez et al., 2008. (Power management in sensor networks)

Example: Vehicle target assignment Resources : Targets Players : Vehicles / Weapons Actions : Possible engagements vehicle 1 Welfare : worth, expected damage and loss. vehicle 2 Welfare no W r (1) communication W r (2) W r (3) vehicle 3 W r (1,2) W r (1,3) range W r (2,3) restriction W r (1,2,3) G. Arslan et al., “Autonomous vehicle-target assignment: a game theoretical formulation,” 2007.

Example: Vehicle target assignment Resources : Targets Players : Vehicles / Weapons Actions : Possible engagements vehicle 1 Welfare : worth, expected damage and loss. vehicle 2 Welfare no W r (1) communication W r (2) W r (3) vehicle 3 W r (1,2) W r (1,3) range W r (2,3) restriction W r (1,2,3) G. Arslan et al., “Autonomous vehicle-target assignment: a game theoretical formulation,” 2007.

Example: Vehicle target assignment Resources : Targets Players : Vehicles / Weapons Actions : Possible engagements vehicle 1 Welfare : worth, expected damage and loss. vehicle 2 Welfare no W r (1) communication W r (2) W r (3) vehicle 3 W r (1,2) W r (1,3) range W r (2,3) restriction W r (1,2,3) G. Arslan et al., “Autonomous vehicle-target assignment: a game theoretical formulation,” 2007.

Example: Vehicle target assignment Resources : Targets Players : Vehicles / Weapons Actions : Possible engagements vehicle 1 Welfare : worth, expected damage and loss. vehicle 2 Welfare Welfare no W r (1) communication W r (1) W r (2) W r (2) W r (3) W r (3) vehicle 3 W r (1,2) W r (1,2) W r (1,3) W r (1,3) range W r (2,3) restriction W r (2,3) W r (1,2,3) W r (1,2,3) G. Arslan et al., “Autonomous vehicle-target assignment: a game theoretical formulation,” 2007.

Example: Vehicle target assignment Resources : Targets Players : Vehicles / Weapons Actions : Possible engagements vehicle 1 Welfare : worth, expected damage and loss. vehicle 2 Welfare Welfare no W r (1) communication W r (1) W r (2) W r (2) W r (3) W r (3) vehicle 3 W r (1,2) W r (1,2) W r (1,3) W r (1,3) range W r (2,3) restriction W r (2,3) W r (1,2,3) W r (1,2,3) Global objective : Maximize sum of welfare (centralized assignment not feasible) G. Arslan et al., “Autonomous vehicle-target assignment: a game theoretical formulation,” 2007.

Utility design Goal: Assign each agent a utility such that the resulting game is desirable - Existence of NE - Efficiency of NE - Locality of information - Tractability - Budget balance Approach: View like a cost sharing problem assignment welfare distributed generates welfare to players U W r ( ) , U distribution rule

Distributed welfare games U W r ( ) , U distribution rule � f r ( i, a r ) W r ( a r ) Utility structure: U i ( a ) = r ∈ a i depends only on Properties of distribution rule: local information 1. f r ( i, a r ) ≥ 0 Budget Balanced: ∈ a i ⇒ f r ( i, a r ) = 0 r / 2. � W ( a ) = U i ( a ) 3. � f r ( i, a r ) ≤ 1 i

Distributed welfare games U W r ( ) , U distribution rule � f r ( i, a r ) W r ( a r ) Utility structure: U i ( a ) = r ∈ a i depends only on Properties of distribution rule: local information Are cost sharing 1. f r ( i, a r ) ≥ 0 methodologies useful in Budget Balanced: ∈ a i ⇒ f r ( i, a r ) = 0 r / 2. designing utilities? � W ( a ) = U i ( a ) 3. � f r ( i, a r ) ≤ 1 i

Equal share 1 � | a r | W r ( a r ) U i ( a i , a − i ) = r ∈ a i NE Budget Complexity exists Balanced Equal share low ** If welfare function is anonymous, then NE exists. (Monderer and Shapley, 1996) W r ( a r ) = W r ( | a r | )

Marginal contribution W r ( a r ) − W r ( a r \ i ) � U i ( a i , a − i ) = r ∈ a i NE Budget Complexity exists Balanced Equal share low Marginal contribution medium (Wolpert and Tumor, 1999)

Shapley value � Sh r ( i, a r ) U i ( a i , a − i ) = r ∈ a i NE Budget Complexity exists Balanced Equal share low Marginal contribution medium Shapley value high (builds upon Hart and Mas-Collell, 1989)

Shapley value � Sh r ( i, a r ) U i ( a i , a − i ) = r ∈ a i NE Budget Complexity exists Balanced � Sh r ( i, N ) = W r ( S ) − W r ( S \ i ) � � ω S Equal share low S ⊆ N : i ∈ S Marginal contribution medium summation over marginal contribution all player subset to player subset Shapley value high intractable for large N (builds upon Hart and Mas-Collell, 1989)

Summary NE Budget Complexity exists Balanced Equal share low Marginal contribution medium Shapley value high Tradeoff: Properties vs. Complexity Is there anything else? No, (weighted) SV only rule that guarantees NE + BB in all games. [Chen, Roughgarden & Valiant, 2008]: Network formation games (uniform) Yes if we restrict attention to special classes of games [JRM & Wierman, 2008]: Not restricted to SV in some settings

Efficiency Can we provide efficiency guarantees for general welfare functions? Price of Anarchy Price of Stability W ( a ne ) W ( a ne ) POA = inf G min POS = inf G max W ( a opt ) W ( a opt ) a ne ∈ G a ne ∈ G worst case performance of any NE worst case performance of best NE (independent of number of game specifics) No. In general a NE can be arbitrarily bad. Yes if welfare is submodular (decreasing marginal welfare)

Submodularity • Submodularity (decreasing marginal welfare) W ( S + s ) − W ( S ) ≥ W ( S ′ + s ) − W ( S ′ ) S ⊂ S ′ ⊂ N • Submodularity can be exploited to improve efficiency range Andreas Krause (Caltech) Sensor coverage Vehicle Target Assignment