A Unified Distributed Algorithm for Non- Games Non-cooperative, - PowerPoint PPT Presentation

A Unified Distributed Algorithm for Non- Games Non-cooperative, Non-convex, and Non-differentiable Jong-Shi Pang ∗ and Meisam Razaviyayn † presented at Workshop on Optimization for Modern Computation Peking University, Beijing, China 10:10 – 10:45 PM, Thursday September 04, 2014 ∗ Department of Industrial and Systems Engineering, University of Southern California, Los Angeles † Visiting Research Assistant, Department of Electrical and Computer Engi- neering, University of Minnesota, Minneapolis 1

The Non-cooperative Game G • An n -player non-cooperative game G wherein each player i = 1 , · · · , n , an- n � ticipating the rivals’ strategy tuple x − i � � x j � n i � = j =1 ∈ X − i � X j , solves the i � = j =1 optimization problem: θ i ( x i , x − i ) minimize x i ∈X i • X i ⊆ R n i is a closed convex set • θ i : Ω → R is a locally Lipschitz continuous and directionally differentiable n � Ω i where each Ω i is an open convex set containing function defined on Ω � i =1 X i • A key structural assumption for convergence of distributed algorithm: each θ i ( x ) = f i ( x )+ g i ( x i ), with f i ( x ), dependent on all players’ strategy profile x � � x i � n i =1 , being twice continuously differentiable but not necessarily convex, and g i ( x i ), dependent on player i ’s strategy profile x i only is convex but not necessarily differentiable. 2

Quasi-Nash equilibrium : Definition and existence � x ∗ ,i � n Definition. A player profile x ∗ � i =1 is a QNE if for every i = 1 , · · · , n : θ i ( • , x ∗ , − i ) ′ ( x ∗ ,i ; x i − x ∗ ,i ) ≥ 0 , ∀ x i ∈ X i . Existence. Suppose each θ i ( x ) = f i ( x ) + g i ( x ) with ∇ x i f i continuously differ- entiable on Ω and g i ( • , x − i ) convex on X i that is compact and convex. Proof by a fixed-point argument applied to the map: n � Φ : x � � x i � n X i �→ Φ( x ) � (Φ i ( x )) n i =1 ∈ X � i =1 ∈ X , where, for i = 1 , · · · , n , i =1 � 2 � z i − x i � 2 � f i ( z i , x − i ) + g i ( z i , x − i ) + α Φ i ( x ) � argmin , z i ∈X i with α > 0 such that the minimand is strongly convex in z i for fixed x − i . Remark. For existence, g i ( x ) can be fully dependent on the player profile x ; but for convergence of distributed algorithm, g i ( x i ) is only player dependent. 3

The unified algorithm The main idea: • Employing player-convex surrogate objective functions and the information from the most current iterate, non-overlapping groups of players update, in parallel, their strategies from the solution of sub-games. • Thus the algorithm is a mixture of the classical block Gauss-Seidel and Jacobi iterations, applied in a way consistent with the game-theoretic setting of the problem. Two key families: • The player groups: σ ν � � � σ ν 1 , · · · , σ ν consists of κ ν pairwise disjoint subsets κ ν of the players’ labels, for some integer κ ν > 0. Players in each group σ ν k solve a sub-game; all such sub-games in iteration ν are solved in parallel. κ ν � σ ν N ν � k not necessarily equal to { 1 , · · · , n } ; k =1 i.e., some players may not update in an iteration. 4

� � κ ν • Given x ν ∈ X , the bivariate surrogate objectives: θ σ ν � i ( x σ ν k ; x ν ) : i ∈ σ ν k k � � κ ν k =1 in lieu of the original objectives { θ i } i ∈ σ ν k =1 . k • The subgames , denoted G σ ν for k = 1 , · · · κ ν : the optimization problems of k ν the players in σ ν k are                       θ σ ν � x i , x σ ν k ; − i x ν minimize ; .   k ���� � �� � i     x i ∈X i       input to subgame   subgame variables     at iteration ν x σ ν k i ∈ σ ν k • The new iterate for a step size τ σ ν k ∈ (0 , 1]   k � x ν ; σ ν x ν +1; σ ν k + τ σ ν  x ν ; σ ν − x ν ; σ ν  . � k k ���� k solution to subgame • Need directional derivative consistency at limit x ∞ of generated sequence: θ i ( • , x ∞ , − i ) ′ ( x ∞ ,i ; x i − x ∞ ,i ) ≥ � k ; − i ; x ∞ ) ′ ( x ∞ ,i ; x i − x ∞ ,i ) , ∀ x i ∈ X i . θ σ t i ( • , x ∞ ,σ t k 5

An illustration. A 10-player game with the grouping: σ ν = { { 1 , 2 } , { 3 , 4 , 5 } , { 6 , 7 , 8 , 9 } } so that κ ν = 3 and N ν = { 1 , · · · , 9 } , leaving out the 10th-player. Players 1 and 2 update their strategies by solving a subgame G { 1 , 2 } defined ν θ { 1 , 2 } θ { 1 , 2 } by the surrogate objective functions � ( • ; x ν ) and � ( • ; x ν ). 1 2 In parallel, players 3, 4, and 5 update their strategies by solving a subgame G { 3 , 4 , 5 } using the surrogate objective functions ν � � θ { 3 , 4 , 5 } θ { 3 , 4 , 5 } θ { 3 , 4 , 5 } � ( • ; x ν ) , � ( • ; x ν ) , � ( • ; x ν ) ; 3 4 5 similarly for players 6 through 9. The 10th player is not performing an update in the current iteration ν ac- cording to the given grouping. 6

Special cases: player groups • Block Jacobi N ν = { 1 , · · · , n } and σ ν k may contains multiple elements. • Point Jacobi κ ν = n ; thus σ ν k = { k } for k = 1 , · · · n : each player i solves an optimization problem: � θ i ( x i ; x ν ) . minimize x i ∈X i • Block Gauss-Seidel κ ν = 1 for all ν : only the players in the block σ ν 1 update their strategies that immediately become the inputs to the new iterate x ν +1 while all other players j �∈ σ ν 1 keep their strategies at the current iterate x ν,j . • Point Gauss-Seidel κ ν = 1 and σ ν 1 is a singleton. • Above are deterministic player groups; also consider randomized player groups: Let { σ 1 , · · · σ K } be a partition of { 1 , · · · , n } . At iteration ν , the subset σ ν ⊆ { σ 1 , . . . , σ K } of player groups is chosen randomly and independently from the previous iterations, so that Pr( σ i ∈ σ ν ) = p σ i > 0 , There is a positive probability p σ i , same at all iterations ν , for the subset σ i of players to be chosen to update their strategies. 7

Special cases: surrogate objectives • Standard convex case Suppose θ i ( • , x − i ) is convex. For i ∈ σ ν k , let k ) + α i 2 � x i − z i � 2 θ σ ν � i ( x σ ν k ; z ) � θ i ( x σ ν k , z − σ ν , for some positive scalar α i k � �� � regularization • Mixed convexity and differentiability Suppose θ i ( • , x − i ) = g i ( • , x − i )+ f i ( • , x − i ), where g i ( • , x − i ) is convex and f i ( • , x − i ) is differentiable. Let � + α i ∇ z j f i ( z ) T ( x j − z j ) 2 � x i − z i � 2 θ σ ν � i ( x σ ν k ; z ) � g i ( x σ ν k , z − σ ν k ) + f i ( z ) + k j ∈ σ ν � k �� � partial linearization � �� � convex in x σ ν k for fixed z • Newton-type quadratic approximation Suppose ∇ x i θ i ( • , x − i ) exists. Let � � ( x j ′ − z j ′ ) T B σ ν k ; j,j ′ ( x j − z j ) ∇ x j θ i ( z ) T ( x j − z j ) + 1 θ σ ν � i ( x σ ν k ; z ) � θ i ( z ) + , k 2 j ∈ σ ν j,j ′ ∈ σ ν � k �� k � quadratic in x σ ν k for fixed z k ; j,j ′ approximates mixed partial derivatives of θ i ( • , z − σ ν k ) w.r.t. x j and x j ′ . B σ ν 8

Convergence analysis Two approaches • Contraction — showing that the sequence { x ν } ∞ ν =1 contracts in the vector sense by means of the assumption of a spectral radius condition of a key matrix • Potential — relying on the existence of a potential function that decreases at each iteration. Think about a system of linear equations: Ax = b • (Generalized) diagonal dominance yields convergence under contraction. • Symmetry of A yields the potential function: P ( x ) � 1 2 x T Ax − b T x . 9

Contraction approach • An integer T > 0 and a fixed family �� σ t � � �� T σ t 1 , · · · , σ t t =1 of index subsets κ t of the players’ labels that partitions { 1 , · · · , n } . • Families of bivariate surrogate functions � � κ t t = θ σ t � � : i ∈ σ t k =1 , for t = 1 , · · · , T, θ k k i θ σ t k ; − i ; z ), the function � such that for every pair ( x σ t i ( • , x σ t k ; − i ; z ) is convex. k k , let G σ t • For each set σ t denote the subgame consisting of the players i ∈ σ t k t k θ σ t with objective functions � i ( • ; z ) for certain (known) iterate z to be specified. k • Let κ ν = κ t and σ ν k = σ t k for ν ≡ t modulo T and for all k = 1 , · · · , κ ν ; thus, θ σ ν θ σ t for each i = 1 , · · · , n , � = � where ν ≡ t modulo T and σ t k is the unique k k i i index set containing i . • Thus, each player i and the members in σ t k will update their strategy tuple exactly once every T iterations through the solution of the subgame G σ t t . k • Finally, we take each step size τ σ ν k = 1. 10