Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Uncoupled dynamics do not lead to NE From Hart and Mas-Colell (2003): Coupled dynamics Uncoupled dynamics x i = F i ( x ; ( u j ) j ∈ N ) ˙ x i = F i ( x ; u i ) ˙ There exist no uncoupled dynamics that guarantee convergence to NE. Exceptions (special families of games): two-player zero-sum games, two-player potential games, and others. There exist uncoupled dynamics converging to correlated equilibria. There exist uncoupled dynamics that are most of the time in ǫ -NE, but do not converge to NE. Therefore: GANs should have coupled dynamics to converge . Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 15 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras (Un)stable fixed points (1/2) A fixed point satisfies: f ( x 0 ) = x 0 Stable (attractive) fixed point Unstable fixed point x , f ( x ) , f ( f ( x )) , f ( f ( f ( x ))) , ... ǫ -neighbourhood of x 0 does not contract to x 0 but diverges. converges to x 0 (Banach FP) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 16 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras (Un)stable fixed points (1/2) A fixed point satisfies: f ( x 0 ) = x 0 Stable (attractive) fixed point Unstable fixed point x , f ( x ) , f ( f ( x )) , f ( f ( f ( x ))) , ... ǫ -neighbourhood of x 0 does not contract to x 0 but diverges. converges to x 0 (Banach FP) Example: golden ratio φ = 1 . 61809339887 ... b = φ → φ 2 − φ − 1 = 0) (solution to a + b = a a Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 16 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras (Un)stable fixed points (1/2) A fixed point satisfies: f ( x 0 ) = x 0 Stable (attractive) fixed point Unstable fixed point x , f ( x ) , f ( f ( x )) , f ( f ( f ( x ))) , ... ǫ -neighbourhood of x 0 does not contract to x 0 but diverges. converges to x 0 (Banach FP) Example: golden ratio φ = 1 . 61809339887 ... b = φ → φ 2 − φ − 1 = 0) (solution to a + b = a a Other solution φ ′ = − 0 . 61809339887 ... Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 16 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras (Un)stable fixed points (1/2) A fixed point satisfies: f ( x 0 ) = x 0 Stable (attractive) fixed point Unstable fixed point x , f ( x ) , f ( f ( x )) , f ( f ( f ( x ))) , ... ǫ -neighbourhood of x 0 does not contract to x 0 but diverges. converges to x 0 (Banach FP) Example: golden ratio φ = 1 . 61809339887 ... b = φ → φ 2 − φ − 1 = 0) (solution to a + b = a a Other solution φ ′ = − 0 . 61809339887 ... Iterative method to find φ : φ t +1 = 1 + 1 φ t Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 16 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras (Un)stable fixed points (1/2) A fixed point satisfies: f ( x 0 ) = x 0 Stable (attractive) fixed point Unstable fixed point x , f ( x ) , f ( f ( x )) , f ( f ( f ( x ))) , ... ǫ -neighbourhood of x 0 does not contract to x 0 but diverges. converges to x 0 (Banach FP) Example: golden ratio φ = 1 . 61809339887 ... b = φ → φ 2 − φ − 1 = 0) (solution to a + b = a a Other solution φ ′ = − 0 . 61809339887 ... Iterative method to find φ : φ t +1 = 1 + 1 φ t Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 16 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras (Un)stable fixed points (2/2) Stable: Unstable: . Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 17 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Takeaways Things to consider: What is the “right” solution concept? Is it robust (stable)? Can we compute it efficiently? On which family of games? Can we approximate it using iterative algorithms? Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 18 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Motivation (1/2) In single objective problems gradient descent (GD) converges to local minima. Even escapes saddle points almost surely! Lee et. al (2017) However, simultaneous/alternating GD is not guaranteed to converge even to local minima! Cyclic behaviour! Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 19 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Motivation (1/2) In single objective problems gradient descent (GD) converges to local minima. Even escapes saddle points almost surely! Lee et. al (2017) However, simultaneous/alternating GD is not guaranteed to converge even to local minima! Cyclic behaviour! Therefore: convergence to local minima is difficult! Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 19 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Motivation (2/2) Simultaneous GD vs SGA � Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 20 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Differentiable games The normal form games we talked about so far were constrained to probability simplex Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 21 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Differentiable games The normal form games we talked about so far were constrained to probability simplex This does not make sense in settings such as GANs - weights are not on simplex! Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 21 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Differentiable games The normal form games we talked about so far were constrained to probability simplex This does not make sense in settings such as GANs - weights are not on simplex! Introduce differentiable games: Set of players N = { 1 , . . . , n } Continuously twice differentiable losses { ℓ i : R d → R } n i =1 NN parameters w = ( w 1 , . . . , w n ) ∈ R d where � n i =1 d i = d Player i controls w i ∈ R d i , and aims to minimize its loss Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 21 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Dynamics, Jacobian Simultaneous gradient is the gradient of the losses w/ respect to the parameters of the respective players: ξ ( w ) = ( ∇ w 1 ℓ 1 , . . . , ∇ w n ℓ n ) ∈ R d . Dynamics of the game mean following − ξ with infinitesimal steps. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 22 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Dynamics, Jacobian Simultaneous gradient is the gradient of the losses w/ respect to the parameters of the respective players: ξ ( w ) = ( ∇ w 1 ℓ 1 , . . . , ∇ w n ℓ n ) ∈ R d . Dynamics of the game mean following − ξ with infinitesimal steps. The Jacobian of a game with dynamics ξ is the ( d × d )-matrix of second-derivatives J ( w ) := ∇ w · ξ ( w ) ⊺ . Concretely: ∇ 2 ∇ 2 ∇ 2 w 1 ℓ 1 w 1 , w 2 ℓ 1 · · · w 1 , w n ℓ 1 ∇ 2 ∇ 2 ∇ 2 w 2 , w 1 ℓ 2 w 2 ℓ 2 · · · w 2 , w n ℓ 2 J ( w ) = . . . . . . ∇ 2 ∇ 2 ∇ 2 · · · w n , w 1 ℓ n w n , w 2 ℓ n w n ℓ n where ∇ 2 w i , w j ℓ k is the ( d i × d j )-block of 2 nd -order derivatives. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 22 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Fixed points and equilibria local Nash equilibrium : NE only within a neighbourhood: if, for all i , there exists a neighborhood U i of w ∗ i such that ℓ i ( w ′ i , w ∗ − i ) ≥ ℓ i ( w ∗ i , w ∗ − i ) for w ′ i ∈ U i . Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 23 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Fixed points and equilibria local Nash equilibrium : NE only within a neighbourhood: if, for all i , there exists a neighborhood U i of w ∗ i such that ℓ i ( w ′ i , w ∗ − i ) ≥ ℓ i ( w ∗ i , w ∗ − i ) for w ′ i ∈ U i . A fixed point w ∗ with ξ ( w ∗ ) = 0 is stable if J ( w ∗ ) � 0 and J ( w ∗ ) is invertible unstable if J ( w ∗ ) ≺ 0 strict saddle if J ( w ∗ ) has an eigenvalue with negative real part (Strict saddles are a subset of unstable fixed points.) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 23 / 65
� Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Fixed points and equilibria local Nash equilibrium : NE only within a neighbourhood: if, for all i , there exists a neighborhood U i of w ∗ i such that ℓ i ( w ′ i , w ∗ − i ) ≥ ℓ i ( w ∗ i , w ∗ − i ) for w ′ i ∈ U i . A fixed point w ∗ with ξ ( w ∗ ) = 0 is stable if J ( w ∗ ) � 0 and J ( w ∗ ) is invertible unstable if J ( w ∗ ) ≺ 0 strict saddle if J ( w ∗ ) has an eigenvalue with negative real part (Strict saddles are a subset of unstable fixed points.) Properties if general-sum: local NE ⇐ ⇒ stable if zero-sum: local NE ⇐ ⇒ stable Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 23 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Decomposition of games (1/2) Any matrix decomposes uniquely as M = S + A where S = 1 2 ( M + M ⊺ ) and A = 1 2 ( M − M ⊺ ) S − S ⊺ ≡ 0 is symmetric and A + A ⊺ ≡ 0 is antisymmetric. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 24 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Decomposition of games (2/2) We can apply this decomposition to the jacobian J ( w ) = S ( w ) + A ( w ) Let’s define: potential game : the jacobian is symmetric, i.e. A ( w ) ≡ 0. hamiltonian game : the jacobian is antisymmetric, i.e. S ( w ) ≡ 0. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 25 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Optimization on the respective game types If game is potential : Well studied games in literature. Simultaneous gradient descent on the losses corresponds to gradient descent on a single function. GD on ξ converges to a fixed point that is a local minimum or a saddle. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 26 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Optimization on the respective game types If game is potential : Well studied games in literature. Simultaneous gradient descent on the losses corresponds to gradient descent on a single function. GD on ξ converges to a fixed point that is a local minimum or a saddle. If game is hamiltonian : Novel contribution of the paper, analyzes properties of hamiltonian games Let H ( w ) := 1 2 � ξ ( w ) � 2 2 (by theorem 4) GD on H converges to stable fixed point Further gradient can be computed as ∇H = A ⊺ ξ Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 26 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Symplectic gradient adjustment We’d like to solve general games, i.e. J ( w ) = S ( w ) + A ( w ) This paper introduces symplectic gradient adjustment (SGA): ξ λ := ξ + λ · A ⊺ ξ . Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 27 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras SGA desiderata SGA satisfies following desiderata: 1 compatible 1 with game dynamics: � ξ λ , ξ � = α 1 · � ξ � 2 ; 2 compatible with potential dynamics: if the game is a potential game then � ξ λ , ∇ φ � = α 2 · �∇ φ � 2 ; 3 compatible with Hamiltonian dynamics: If the game is Hamiltonian then � ξ λ , ∇H� = α 3 · �∇H� 2 ; 4 attracted to stable equilibria: in neighborhoods where S ≻ 0, require θ ( ξ λ , ∇H ) ≤ θ ( ξ , ∇H ); 5 repelled by unstable equilibria: in neighborhoods where S ≺ 0, require θ ( ξ λ , ∇H ) ≥ θ ( ξ , ∇H ). for some α 1 , α 2 , α 3 > 0. 1 Two nonzero vectors are compatible if they have positive inner product. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 28 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras SGA pseudocode Input: losses L = { ℓ i } n i =1 , weights W = { w i } n i =1 � � ξ ← gradient ( ℓ i , w i ) for ( ℓ i , w i ) ∈ ( L , W ) A ⊺ ξ ← get sym adj ( L , W ) // appendix A if align then gradient ( 1 � 2 � ξ � 2 , w ) for w ∈ W ) � ∇H ← � � 1 // ǫ = 1 λ ← sign d � ξ , ∇H�� A ⊺ ξ , ∇H� + ǫ 10 else λ ← 1 end if Output: ξ + λ · A ⊺ ξ // plug into any optimizer Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 29 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras SGA (pytorch) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 30 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Experiment: higher learning rates learning rate 0.01 0.032 0.1 GRADIENT DESCENT SGA SGA allows faster and more robust convergence to stable fixed points than vanilla gradient descent in the presence of ’rotational forces’, by bending the direction of descent towards the fixed point. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 31 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Experiment: convergence with various learning rates OMD LOSS AFTER 250 STEPS OMD STEPS TO CONVERGE SGA SGA LEARNING RATE LEARNING RATE Comparison of SGA with optimistic mirror descent. SGA with λ = 1 (Left): iterations to convergence, with maximum 250 iters. (Right): average absolute value of losses over the last 10 iterations, i.e. iterations 240-250 Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 32 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Experiment: convergence to all modes (1/2) Ground truth: Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 33 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Experiment: convergence to all modes (2/2) GRADIENT DESCENT learning rate 1e-4 SGA without ALIGNMENT learning rate 9e-5 SGA with ALIGNMENT learning rate 9e-5 CONSENSUS OPTIMIZATION learning rate 9e-5 CONSENSUS OPTIMIZATION with ALIGNMENT learning rate 9.25e-5 Iteration: 2000 4000 6000 8000 Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 34 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras My experiments: learning in NFGs - 2x2 game Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 35 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras My experiments: learning in NFGs - 10x10 game Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 36 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Notes Adjustment update independently discovered in Global Convergence to the Equilibrium of GAS using Variational Inequalities , I. Gemp, S. Mahadevan Arxiv � (called Crossing-the-Curl) Things I’ve left out: Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 37 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Notes Adjustment update independently discovered in Global Convergence to the Equilibrium of GAS using Variational Inequalities , I. Gemp, S. Mahadevan Arxiv � (called Crossing-the-Curl) Things I’ve left out: Sign and magnitude of the adjustment λ Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 37 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Notes Adjustment update independently discovered in Global Convergence to the Equilibrium of GAS using Variational Inequalities , I. Gemp, S. Mahadevan Arxiv � (called Crossing-the-Curl) Things I’ve left out: Sign and magnitude of the adjustment λ Properties between Hamiltonian and zero-sum games Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 37 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Notes Adjustment update independently discovered in Global Convergence to the Equilibrium of GAS using Variational Inequalities , I. Gemp, S. Mahadevan Arxiv � (called Crossing-the-Curl) Things I’ve left out: Sign and magnitude of the adjustment λ Properties between Hamiltonian and zero-sum games Jacobian-vector product can be computed efficiently Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 37 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Notes Adjustment update independently discovered in Global Convergence to the Equilibrium of GAS using Variational Inequalities , I. Gemp, S. Mahadevan Arxiv � (called Crossing-the-Curl) Things I’ve left out: Sign and magnitude of the adjustment λ Properties between Hamiltonian and zero-sum games Jacobian-vector product can be computed efficiently Type consistency check of the game reveals if gradient adjustment is needed Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 37 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Notes Adjustment update independently discovered in Global Convergence to the Equilibrium of GAS using Variational Inequalities , I. Gemp, S. Mahadevan Arxiv � (called Crossing-the-Curl) Things I’ve left out: Sign and magnitude of the adjustment λ Properties between Hamiltonian and zero-sum games Jacobian-vector product can be computed efficiently Type consistency check of the game reveals if gradient adjustment is needed Relation to differential and symplectic geometry and Hodge decomposition Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 37 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Takeaways Multiobjective optimization: Simultaneous gradient is not guaranteed to converge to local minima in general games (only in potential games). Causes mode collapse and mode-hopping Symplectic gradient adjustment Simple plug-n-play adjustment for optimization Supports multiplayer settings n ≥ 2 Implementation for Tensorflow/Pytorch available Computes stable fixed points which are not necessarily local NE Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 38 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Motivation Applying latent optimization leads to more diverse image generation. ( a ) ( b ) Figure: Samples from BigGAN-deep ( a ) and LOGAN ( b ) with similarly high IS. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 39 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Gradient update in GANs min max E x ∼ p ( x ) [ h D ( D ( x ; θ D ))] + E z ∼ p ( z ) [ h G ( D ( G ( z ; θ G ) ; θ D ))] θ D θ G Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 40 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Gradient update in GANs min max E x ∼ p ( x ) [ h D ( D ( x ; θ D ))] + E z ∼ p ( z ) [ h G ( D ( G ( z ; θ G ) ; θ D ))] θ D θ G LOGAN settings h D ( t ) = − t , h G ( t ) = t : min max E x ∼ p ( x ) [ − D ( x ; θ D ))] + E z ∼ p ( z ) [ D ( G ( z ; θ G ) ; θ D )] θ D θ G Notation: f ( z ) := D ( G ( z ; θ G ) ; θ D ) Gradient update for discriminator and generator: � T � ∂ f ( z ) , − ∂ f ( z ) ξ = (1) ∂θ D ∂θ G Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 40 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras SGA for GANs SGA has extra term: ξ λ := ξ + λ · A ⊺ ξ Applying SGA to GANs: � ∂ 2 f ( z ) � T ∂ f ( z ) ∂ f ( z ) + λ ∂θ D ∂θ G ∂θ D ∂θ G ξ λ := � ∂ 2 f ( z ) � T − ∂ f ( z ) ∂ f ( z ) + λ ∂θ G ∂θ D ∂θ G ∂θ D Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 41 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Latent optimized sources Idea: instead of using z , use latent optimized z ′ := z + ∆ z Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 42 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Computing ∆ z Two approaches to compute ∆ z : Gradient Descent: ∆ z = α g Natural Gradient Descent (approximation of 2nd order method): α ∆ z = β + � g � 2 g with g = ∂ f ( z ) ∂ z . Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 43 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Latent Optimised GANs with Automatic Differentiation Input: data distribution p ( x ) latent distribution p ( z ) D ( · ; θ D ), G ( · ; θ G ) learning rate α , batch size N repeat Initialise discriminator and generator parameters θ D , θ G for i = 1 to N do Sample z ∼ p ( z ), x ∼ p ( x ) Compute the gradient ∂ D ( G ( z )) and obtain ∆ z using GD or NGD ∂ z Compute the optimized latent z ′ ← [ z + ∆ z ] a Compute generator loss L ( i ) G = − D ( G ( z ′ )) Compute discriminator loss L ( i ) D = D ( G ( z ′ )) − D ( x ) end for i =1 L ( i ) i =1 L ( i ) Compute batch losses L G = 1 � N G and L D = 1 � N N N D Update θ D and θ G with the gradients ∂ L D ∂θ D , ∂ L G ∂θ G until reaches the maximum training steps a [ · ] indicates clipping the value between − 1 and 1 Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 44 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Latent optimized dynamics How does z ′ = z + ∆ z = z + α ∂ f ( z ) ∂ z � T � ∂ f ( z ) ∂θ D , − ∂ f ( z ) change the original dynamics ξ = ? ∂θ G Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 45 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Latent optimized dynamics How does z ′ = z + ∆ z = z + α ∂ f ( z ) ∂ z � T � ∂ f ( z ) ∂θ D , − ∂ f ( z ) change the original dynamics ξ = ? ∂θ G � T � T � � ∂ 2 f ( z ) ∂ f ( z ′ ) ∂ f ( z ′ ) ∂ f ( z ′ ) ∂ f ( z ′ ) ∂ ∆ z + + α ∂θ D ∂θ D ∂ ∆ z ∂θ D ∂ z ∂θ D ∂ z ′ ξ ′ = = � T � T � � ∂ 2 f ( z ) − ∂ f ( z ′ ) ∂ f ( z ′ ) − ∂ f ( z ′ ) ∂ f ( z ′ ) ∂ ∆ z − − α ∂θ G ∂θ G ∂ ∆ z ∂θ G ∂ z ∂θ G ∂ z ′ Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 45 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Comparison of the two dynamics SGA: Latent optimization: � ∂ 2 f ( z ) � T � T ∂ f ( z ) ∂ f ( z ) � ∂ 2 f ( z ) ∂ f ( z ′ ) ∂ f ( z ′ ) + λ + α ∂θ D ∂θ G ∂θ D ∂θ G ∂θ D ∂ z ∂θ D ∂ z ′ ξ ′ = ξ λ = � ∂ 2 f ( z ) � T � T − ∂ f ( z ) ∂ f ( z ) � ∂ 2 f ( z ) − ∂ f ( z ′ ) ∂ f ( z ′ ) + λ − α ∂θ G ∂θ D ∂θ G ∂θ D ∂ z ∂θ G ∂ z ′ ∂θ G Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 46 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Comparison of the two dynamics SGA: Latent optimization: � ∂ 2 f ( z ) � T � T ∂ f ( z ) ∂ f ( z ) � ∂ 2 f ( z ) ∂ f ( z ′ ) ∂ f ( z ′ ) + λ + α ∂θ D ∂θ G ∂θ D ∂θ G ∂θ D ∂ z ∂θ D ∂ z ′ ξ ′ = ξ λ = � ∂ 2 f ( z ) � T � T − ∂ f ( z ) ∂ f ( z ) � ∂ 2 f ( z ) − ∂ f ( z ′ ) ∂ f ( z ′ ) + λ − α ∂θ G ∂θ D ∂θ G ∂θ D ∂ z ∂θ G ∂ z ′ ∂θ G Approximates SGA using only second-order derivatives with respect to the latent z and parameters of the discriminator and generator separately . Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 46 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Comparison of the two dynamics SGA: Latent optimization: � ∂ 2 f ( z ) � T � T ∂ f ( z ) ∂ f ( z ) � ∂ 2 f ( z ) ∂ f ( z ′ ) ∂ f ( z ′ ) + λ + α ∂θ D ∂θ G ∂θ D ∂θ G ∂θ D ∂ z ∂θ D ∂ z ′ ξ ′ = ξ λ = � ∂ 2 f ( z ) � T � T − ∂ f ( z ) ∂ f ( z ) � ∂ 2 f ( z ) − ∂ f ( z ′ ) ∂ f ( z ′ ) + λ − α ∂θ G ∂θ D ∂θ G ∂θ D ∂ z ∂θ G ∂ z ′ ∂θ G Approximates SGA using only second-order derivatives with respect to the latent z and parameters of the discriminator and generator separately . The second order terms involving parameters of both the discriminator and the generator – which are extremely expensive to compute – are not used. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 46 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Comparison of the two dynamics SGA: Latent optimization: � ∂ 2 f ( z ) � T � T ∂ f ( z ) ∂ f ( z ) � ∂ 2 f ( z ) ∂ f ( z ′ ) ∂ f ( z ′ ) + λ + α ∂θ D ∂θ G ∂θ D ∂θ G ∂θ D ∂ z ∂θ D ∂ z ′ ξ ′ = ξ λ = � ∂ 2 f ( z ) � T � T − ∂ f ( z ) ∂ f ( z ) � ∂ 2 f ( z ) − ∂ f ( z ′ ) ∂ f ( z ′ ) + λ − α ∂θ G ∂θ D ∂θ G ∂θ D ∂ z ∂θ G ∂ z ′ ∂θ G Approximates SGA using only second-order derivatives with respect to the latent z and parameters of the discriminator and generator separately . The second order terms involving parameters of both the discriminator and the generator – which are extremely expensive to compute – are not used. For latent z ’s with dimensions typically used in GANs, the second order terms can be computed efficiently. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 46 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Comparison of the two dynamics SGA: Latent optimization: � ∂ 2 f ( z ) � T � T ∂ f ( z ) ∂ f ( z ) � ∂ 2 f ( z ) ∂ f ( z ′ ) ∂ f ( z ′ ) + λ + α ∂θ D ∂θ G ∂θ D ∂θ G ∂θ D ∂ z ∂θ D ∂ z ′ ξ ′ = ξ λ = � ∂ 2 f ( z ) � T � T − ∂ f ( z ) ∂ f ( z ) � ∂ 2 f ( z ) − ∂ f ( z ′ ) ∂ f ( z ′ ) + λ − α ∂θ G ∂θ D ∂θ G ∂θ D ∂ z ∂θ G ∂ z ′ ∂θ G Approximates SGA using only second-order derivatives with respect to the latent z and parameters of the discriminator and generator separately . The second order terms involving parameters of both the discriminator and the generator – which are extremely expensive to compute – are not used. For latent z ’s with dimensions typically used in GANs, the second order terms can be computed efficiently. In short, latent optimisation efficiently couples the gradients of the D and G, as prescribed by SGA, but using the much lower-dimensional latent source z . Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 46 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Evaluation Evaluation of GANs is hard Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 47 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Evaluation Evaluation of GANs is hard If we could specify it well enough, it could become the objective! Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 47 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Evaluation Evaluation of GANs is hard If we could specify it well enough, it could become the objective! Best response against a fixed discriminator may not be what we want. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 47 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Evaluation Evaluation of GANs is hard If we could specify it well enough, it could become the objective! Best response against a fixed discriminator may not be what we want. Scores used in LOGAN: Inception Score (IS) a measure the “objectness” of a generated image, computed by model with Inception architecture (Salimans 2016) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 47 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Evaluation Evaluation of GANs is hard If we could specify it well enough, it could become the objective! Best response against a fixed discriminator may not be what we want. Scores used in LOGAN: Inception Score (IS) a measure the “objectness” of a generated image, computed by model with Inception architecture (Salimans 2016) Fr´ echet Inception Distance (FID) todo evaluate the similarity between two dataset of images (Heusel 2017) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 47 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Experiments: IS vs FID . BigGAN: architecture based on residual blocks regularisation mechanisms and self-attention Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 48 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Experiments: IS vs FID . BigGAN: architecture based on residual blocks regularisation mechanisms and self-attention LOGAN improves the adversarial dynamics during training. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 48 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Experiments: IS vs FID . BigGAN: architecture based on residual blocks regularisation mechanisms and self-attention LOGAN improves the adversarial dynamics during training. No need to optimize latents for evaluation. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 48 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 49 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Samples - High IS (points C,D) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 50 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Samples - High IS (points C,D) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 51 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Samples - High IS (points C,D) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 52 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Samples - High IS (points C,D) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 53 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Samples - Low FID (points A,B) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 54 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Samples - Low FID (points A,B) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 55 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Samples - Low FID (points A,B) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 56 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Samples - Low FID (points A,B) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 57 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Negative review The LOGAN paper was not accepted to ICLR 2020. Main criticism: authors do not compare to SGA. The reviewers claim that computing the Hessian vector product is not that expensive and they should’ve done it. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 58 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Negative review The LOGAN paper was not accepted to ICLR 2020. Main criticism: authors do not compare to SGA. The reviewers claim that computing the Hessian vector product is not that expensive and they should’ve done it. Authors did not cite concurrent submission to ICLR 2020 :) Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 58 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Notes I did not talk about: Relation with Unrolled GANs, and that SGA can be seen as approximating Unrolled GANs Relation with stochastic approximation with two timescales. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 59 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Takeaways Mainly experimental paper. Introduces a simplfication of the SGA update. + Coupling the generator/discriminator via latent optimization improves sample quality. - Latent optimization has a higher cost per iteration (they claim about 3x slower). Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 60 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Conclusion Multiobjective optimization is an area of game theory. Game theory offers various solution concepts. It’s not clear what is the “best” solution concept. It’s hard to converge to Nash equilibria. We might be satisfied with stable fixed points. It’s possible to update simultaneous gradient descent to converge to stable FP. This is called symplectic gradient adjustment (SGA). SGA can be applied to GANs of any architecture. LOGAN is an approximation of SGA to GANs. LOGAN generates higher quality and more diverse images. Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 61 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Questions? Hannan consistency of SGA? Convergence to correlated equilibria? Relation to algorithms like exploitability descent? Find a simple failure case: finds local NE but not a global NE What GANs (do not) use coupling and how it influences performance? Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 62 / 65
Introduction Background Differentiable Game Mechanics LOGAN Conclusion Extras Strategy profile σ = ( σ i , σ − i ) is an ǫ -Nash equilibrium ( ǫ -NE) if ( ∀ i ∈ N ) : u i ( σ ) ≥ max u i ( σ ′ i , σ − i ) − ǫ. σ ′ i ∈ Σ i Michal ˇ Sustr Reading group: Latent Optimized GANs January 9th, 2020 63 / 65
Recommend
More recommend
