Constant Curvature Graph Convolutional Networks Gregor Gary - PowerPoint PPT Presentation

Constant Curvature Graph Convolutional Networks Gregor Gary Octavian Bachmann B´ ecigneul Ganea ETH Z¨ urich MIT MIT

Overview

Overview • Embeddings of graphs into hyperbolic and spherical space and their products

Overview • Embeddings of graphs into hyperbolic and spherical space and their products • Extend gyrovector framework to spherical geometry and provide a unifying formalism

Overview • Embeddings of graphs into hyperbolic and spherical space and their products • Extend gyrovector framework to spherical geometry and provide a unifying formalism • Introduce graph neural networks producing embeddings in product spaces

Overview • Embeddings of graphs into hyperbolic and spherical space and their products • Extend gyrovector framework to spherical geometry and provide a unifying formalism • Introduce graph neural networks producing embeddings in product spaces • Differentiable transitions in geometry during training in each component

Graphs

Graphs • Lots of data available in the form of graphs (social networks, railway tracks, phylogenetic trees etc.)

Graphs • Lots of data available in the form of graphs (social networks, railway tracks, phylogenetic trees etc.) • Node set V = { 1 , . . . , n } and adjacency matrix A ∈ R n × n

Where to Embed Graphs?

Where to Embed Graphs? • Euclidean geometry not suitable for many graphs

Where to Embed Graphs? • Euclidean geometry not suitable for many graphs • Graph distance d G ( i , j ) = ”Shortest path from i to j” not respected in Euclidean embedding

Where to Embed Graphs? • Euclidean geometry not suitable for many graphs • Graph distance d G ( i , j ) = ”Shortest path from i to j” not respected in Euclidean embedding • Arbitrary low distortion in spherical and hyperbolic space

Non-Euclidean Geometry

Non-Euclidean Geometry • Focus on constant sectional curvature manifolds

Non-Euclidean Geometry • Focus on constant sectional curvature manifolds • Well-studied in the field of Differential Geometry

Non-Euclidean Geometry • Focus on constant sectional curvature manifolds • Well-studied in the field of Differential Geometry • Computationally attractive expressions for distance, exponential map etc.

Hyperbolic Space as Poincar´ e Ball

Hyperbolic Space as Poincar´ e Ball • H n = { x : || x || 2 ≤ 1 √ c } with curvature − c equipped with 4 Riemannian tensor g c x = (1 − c || x || 2 ) 2 1

Hyperbolic Space as Poincar´ e Ball • H n = { x : || x || 2 ≤ 1 √ c } with curvature − c equipped with 4 Riemannian tensor g c x = (1 − c || x || 2 ) 2 1 • Projection of hyperboloid

Hyperbolic Space as Poincar´ e Ball • H n = { x : || x || 2 ≤ 1 √ c } with curvature − c equipped with 4 Riemannian tensor g c x = (1 − c || x || 2 ) 2 1 • Projection of hyperboloid � � 2 c || x − y || 2 √ c cosh − 1 1 • d c H ( x , y ) = 1 + 2 ( 1 2 )( 1 2 ) c −|| x || 2 c −|| y || 2 Heatmap of d κ Projection of hyperboloid [4] H

Gyrospace Structure

Gyrospace Structure • Next best thing to a vector space

Gyrospace Structure • Next best thing to a vector space • Vector addition x + y �→ x ⊕ c y

Gyrospace Structure • Next best thing to a vector space • Vector addition x + y �→ x ⊕ c y • Scalar multiplication r x �→ r ⊗ c x

Gyrospace Structure • Next best thing to a vector space • Vector addition x + y �→ x ⊕ c y • Scalar multiplication r x �→ r ⊗ c x • Geodesic γ x − → y ( t ) = x ⊕ c ( t ⊗ c ( − x ⊕ c y ))

Spherical Space as Stereographic Projection

Spherical Space as Stereographic Projection • Stereographic projection of S d +1 ∼ = R d + g c x where g c 4 x = (1+ c || x || 2 ) 2 1

Spherical Space as Stereographic Projection • Stereographic projection of S d +1 ∼ = R d + g c x where g c 4 x = (1+ c || x || 2 ) 2 1 � � 2 c || x − y || 2 • d c √ c cos − 1 1 2 S ( x , y ) = 1 + ( 1 2 )( 1 c + || x || 2 c + || y || 2 2 )

Our Contributions: 1) Unified Formalism

Our Contributions: 1) Unified Formalism • κ - stereographic model for any κ ∈ R : κ = { x ∈ R d | − κ � x � 2 st d 2 < 1 }

Our Contributions: 1) Unified Formalism • κ - stereographic model for any κ ∈ R : κ = { x ∈ R d | − κ � x � 2 st d 2 < 1 } R d st d κ (1 − 2 κ x T y − κ || y || 2 ) x +(1+ κ || x || 2 ) y x ⊕ κ y x + y 1 − 2 κ x T y + κ 2 || x || 2 || y || 2 tan κ ( r · tan − 1 κ || x || ) x r ⊗ κ x r x || x || γ x → y ( t ) x + t ( y − x ) x ⊕ κ ( t ⊗ κ ( − x ⊕ κ y ))

Our Contributions: 1) Unified Formalism • κ - stereographic model for any κ ∈ R : κ = { x ∈ R d | − κ � x � 2 st d 2 < 1 } R d st d κ (1 − 2 κ x T y − κ || y || 2 ) x +(1+ κ || x || 2 ) y x ⊕ κ y x + y 1 − 2 κ x T y + κ 2 || x || 2 || y || 2 tan κ ( r · tan − 1 κ || x || ) x r ⊗ κ x r x || x || γ x → y ( t ) x + t ( y − x ) x ⊕ κ ( t ⊗ κ ( − x ⊕ κ y )) • More unifying expressions for distance , exponential map etc. in our paper!

Our Contributions: 2) Matrix Multiplications

Our Contributions: 2) Matrix Multiplications κ , W ∈ R d × k and A ∈ R n × n • Embeddings X where X i • ∈ st d

Our Contributions: 2) Matrix Multiplications κ , W ∈ R d × k and A ∈ R n × n • Embeddings X where X i • ∈ st d • Right matrix multiplication XW acts on columns X • i Thus lift to tangent space at zero: ( X ⊗ κ W ) i • = exp κ 0 ((log κ 0 ( X ) W ) i • )

Our Contributions: 2) Matrix Multiplications κ , W ∈ R d × k and A ∈ R n × n • Embeddings X where X i • ∈ st d • Right matrix multiplication XW acts on columns X • i Thus lift to tangent space at zero: ( X ⊗ κ W ) i • = exp κ 0 ((log κ 0 ( X ) W ) i • ) • Introduced in [2], we extended it to spherical spaces

Our Contributions: 2) Matrix Multiplications • Left matrix multiplication AX acts on rows X i • : ( AX ) i • = A i 1 X 1 • + · · · + A in X n •

Our Contributions: 2) Matrix Multiplications • Left matrix multiplication AX acts on rows X i • : ( AX ) i • = A i 1 X 1 • + · · · + A in X n • • Idea: Reduce problem of linear combination to definition of a non-euclidean midpoint

Our Contributions: 2) Matrix Multiplications • Leverage gyromidpoint for hyperbolic space and extend it to st d κ : � n � m κ ( x 1 , · · · , x n ; α ) = 1 α i λ κ � x i 2 ⊗ κ x j − 1) x i � n j =1 α j ( λ κ i =1

Our Contributions: 2) Matrix Multiplications • Leverage gyromidpoint for hyperbolic space and extend it to st d κ : � n � m κ ( x 1 , · · · , x n ; α ) = 1 α i λ κ � x i 2 ⊗ κ x j − 1) x i � n j =1 α j ( λ κ i =1 • Define left matrix multiplication row-wise: � ( A ⊠ κ X ) i • := ( A ij ) ⊗ κ m κ ( X 1 • , · · · , X n • ; A i • ) j

Our Contributions: 2) Matrix Multiplications • Leverage gyromidpoint for hyperbolic space and extend it to st d κ : � n � m κ ( x 1 , · · · , x n ; α ) = 1 α i λ κ � x i 2 ⊗ κ x j − 1) x i � n j =1 α j ( λ κ i =1 • Define left matrix multiplication row-wise: � ( A ⊠ κ X ) i • := ( A ij ) ⊗ κ m κ ( X 1 • , · · · , X n • ; A i • ) j • Same scaling behaviour: d κ ( 0 , r ⊗ κ x ) = r · d κ ( 0 , x )

Gyromidpoint for Varying Curvature

Our Contributions: 3) Differentiable Interpolation

Our Contributions: 3) Differentiable Interpolation → 0 ± • All quantities recover their Euclidean counterpart for κ −

Our Contributions: 3) Differentiable Interpolation → 0 ± • All quantities recover their Euclidean counterpart for κ − • We proved an even stronger result:

Our Contributions: 3) Differentiable Interpolation → 0 ± • All quantities recover their Euclidean counterpart for κ − • We proved an even stronger result: Differentiability of st d κ w.r.t. κ around 0 The first order derivatives at 0 − and 0 + w.r.t. to κ of all the mentioned quantities exist and are equal .

Constant Curvature Graph Convolutional Networks Gregor Gary - PowerPoint PPT Presentation

Constant Curvature Graph Convolutional Networks Gregor Gary Octavian Bachmann B ecigneul Ganea ETH Z urich MIT MIT Overview Overview Embeddings of graphs into hyperbolic and spherical space and their products Overview

On the Experimental transferability of Spectral Graph Convolutional Networks Masters project

EMNLP 2017 Copenhagen Contributions } Syntactic Graph Convolutional Networks }

Graph Convolutional Networks

Simplifying Graph Convolutional Networks Amauri Holanda Felix Wu* Tianyi Zhang* Christopher

Convolutional Kernel Networks for Graph-Structured Data Dexiong Chen 1 Laurent Jacob 2 Julien

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

When Does Self-Supervision Help Graph Convolutional Networks? Yuning You * , Tianlong Chen * ,

Graph Representation Learning with Graph Convolutional Networks Jure Leskovec Networks: Common

Guardian : Evaluating Trust in Online Social Networks with Graph Convolutional Networks Wanyu Lin

Inference Networks, Graph Convolutional Networks Greg Mori School of Computing Science Simon

L 2 -GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks Yuning You * ,

Extracting and Modeling Relations with Graph Convolutional Networks Ivan Titov with Diego

High Performance Graph Convolutional Networks with Applications in Testability Analysis Yuzhe Ma 1

Constant mean curvature surfaces in homogeneous manifolds Beno t Daniel August 29, 2012

Embedded constant mean curvature tori in the three-sphere Haizhong Li (Tsinghua University)

Batched Sparse Matrix Multiplication for Accelerating Graph Convolutional Networks Yusuke Nagasaka

Kipf, T., Welling, M.: Semi-Supervised Classification with Graph Convolutional Networks Radim

Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Semi-supervised Geolocation via Graph Convolutional Networks Afshin Rahimi, Trevor Cohn and Tim

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning Qimai Li, Zhichao

Graph Convolutional Networks (GCNs) Dimitris Papatheodorou Aalto University

Some rigidity results for complete spacelike hypersurfaces with constant mean curvature in

Almost K ahler 4-manifolds of Constant Holomorphic Sectional Curvature are K ahler M.

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)