Deeper Insights into Graph Convolutional Networks for - PowerPoint PPT Presentation

Oct 11, 2023 •729 likes •880 views

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning Qimai Li, Zhichao Han, Xiao-Ming Wu Department of Computing The Hong Kong Polytechnic University Supervised Learning Tons of labeled data A good model 2 Image

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning Qimai Li, Zhichao Han, Xiao-Ming Wu Department of Computing The Hong Kong Polytechnic University
Supervised Learning Tons of labeled data A good model 2 Image via “https://www.linkedin.com/pulse/deep-learning-aviation-francis-j-duque”
Semi-Supervised Learning (SSL) How unlabeled data helps? Decision Boundary Better Decision Boundary 3 Image via “https://en.wikipedia.org/wiki/Semi-supervised_learning”
How to Leverage Unlabeled Data Node -> Document Edge -> Citation Link 4 Image via https://www.cwts.nl/media/images/content/b515d3b727bc41fe7e858df0ffd062bf_large.png
Graph Convolutional Networks (Kipf & Welling, ICLR, 2017) Layer-wise propagation rule: ! "#$ = & '! " Θ " Convolution layer: Projection layer: preprocessed fully connected networks adjacency matrix GCNs for semi-supervised classification: ) = softmax '& '12 3 2 $ 5
Why GCNs Work ! " = $ %&Θ ( Convolution layer: Projection layer: preprocessed fully connected adjacency matrix networks 6
Laplacian Smoothing ! "#$ = & '! " Θ " Smoothing 7
Limitations of GCNs (1) ! "#$ = & '! " Θ " Localized filter Labeled instance Unlabeled instance Instance adjacent to a labeled instance Need to stack many layers to explore global graph topology when labeled data is few - overfitting 8
Limitations of GCNs (2) Need additional labeled data for model selection 9
Our Solutions (1) • Co-train a GCN with a random walk model – Use random walks to explore global topology – Extend the labeled set with high-confidence predictions by the random walk model Labeled instance Unlabeled instance Pseudo labeled instance 10
Our Solutions (2) • Self-training – Extend the labeled set with high-confidence predictions by a pre-trained GCN • Union of Self-training and Co-training – Add to the diversity of pseudo labels • Intersection of Self-training and Co-training – Get more accurate pseudo labels 11
Experimental Results Significant improvements on 3 citation networks 12
Summary • Contributions – Principled understanding of the working mechanisms and limitations of GCNs for SSL – Solutions to improve GCNs • Future directions – Designing more powerful convolution filters – Techniques for training GCNs 13

Recommend

On the Experimental transferability of Spectral Graph Convolutional Networks Masters project

On the Experimental transferability of Spectral Graph Convolutional Networks Masters project presentation 6/ 7/ 2020 Axel Nilsson Outline 1. Introduction - Spectral graph convolutional networks - ChebNet 2. Benchmarking -

317 views • 18 slides

EMNLP 2017 Copenhagen Contributions } Syntactic Graph Convolutional Networks }

Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling Diego Marcheggiani and Ivan Titov University of Amsterdam University of Edinburgh EMNLP 2017 Copenhagen Contributions } Syntactic Graph Convolutional

865 views • 55 slides

CS4501: Introduction to Computer Vision Deeper Convolutional Neural Network Architectures Last

CS4501: Introduction to Computer Vision Deeper Convolutional Neural Network Architectures Last Class Neural Networks multilayer perceptron model (MLP) Backpropagation Convolutional Neural Networks Todays Class More on

772 views • 41 slides

Graph Convolutional Networks

Graph_Convolutional_Networks slides 12/11/19, 8(56 AM Thanks for joining me for a presentation on ... Graph Convolutional Networks http://127.0.0.1(8000/Graph_Convolutional_Networks.slides.html?print-pdf#/ Page 1 of 30

676 views • 30 slides

Simplifying Graph Convolutional Networks Amauri Holanda Felix Wu* Tianyi Zhang* Christopher

Cornell University Simplifying Graph Convolutional Networks Amauri Holanda Felix Wu* Tianyi Zhang* Christopher Fifty Tao Yu Kilian Q. Weinberger de Souza Jnior* *: Equal contribution Cornell University Graph Convolutional Networks

847 views • 21 slides

Convolutional neural networks are good at representation learning Image Object Semantic

Convolutional neural networks are good at representation learning Image Object Semantic Face Pose classification detection segmentation alignment estimation 2 deeper wider finer Deeper - more layers Wider - more

877 views • 70 slides

Convolutional Kernel Networks for Graph-Structured Data Dexiong Chen 1 Laurent Jacob 2 Julien

Convolutional Kernel Networks for Graph-Structured Data Dexiong Chen 1 Laurent Jacob 2 Julien Mairal 1 1 Inria Grenoble 2 CNRS/LBBE Lyon ICML 2020 Dexiong Chen Graph Convolutional Kernel Networks 1 / 15 Graph-structured data are ubiquitous (a)

865 views • 28 slides

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks Wei-Lin Chiang 1 , Xuanqing Liu 2 , Si Si 3 , Yang Li 3 , Samy Bengio 3 , Cho-Jui Hsieh 23 1 National Taiwan University, 2 UCLA, 3 Google Research Graph

616 views • 23 slides

When Does Self-Supervision Help Graph Convolutional Networks? Yuning You * , Tianlong Chen * ,

When Does Self-Supervision Help Graph Convolutional Networks? Yuning You * , Tianlong Chen * , Zhangyang Wang, Yang Shen Texas A&M University * Equal Contribution Department of Electrical and Computer Engineering 1 This work was presented

462 views • 10 slides

Graph Representation Learning with Graph Convolutional Networks Jure Leskovec Networks: Common

Graph Representation Learning with Graph Convolutional Networks Jure Leskovec Networks: Common Language Movie 1 friend Actor 2 co-worker Actor 1 Mary Peter Actor 4 Movie 3 Tom Movie 2 friend brothers Actor 3 Albert Protein 2

752 views • 57 slides

Guardian : Evaluating Trust in Online Social Networks with Graph Convolutional Networks Wanyu Lin

Guardian : Evaluating Trust in Online Social Networks with Graph Convolutional Networks Wanyu Lin , Zhaolin Gao, Baochun Li University of Toronto Almost 4.57 billion people were active internet users as of April 2020. Statista Social trust

733 views • 35 slides

Inference Networks, Graph Convolutional Networks Greg Mori School of Computing Science Simon

Inference Networks, Graph Convolutional Networks Greg Mori School of Computing Science Simon Fraser University Outline Scene Image annotation with label hierarchies outdoor outdoor indoor man-made natural house sports man-made

682 views • 47 slides

L 2 -GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks Yuning You * ,

L 2 -GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks Yuning You * , Tianlong Chen * , Zhangyang Wang, Yang Shen Texas A&M University * Equal Contribution Department of Electrical and Computer Engineering 1

215 views • 8 slides

Extracting and Modeling Relations with Graph Convolutional Networks Ivan Titov with Diego

Extracting and Modeling Relations with Graph Convolutional Networks Ivan Titov with Diego Marcheggiani, Michael Schlichtkrull, Thomas Kipf, Max Welling, Rianne van den Berg and Peter Bloem 1 Inferring missing facts in knowledge bases: link

827 views • 66 slides

High Performance Graph Convolutional Networks with Applications in Testability Analysis Yuzhe Ma 1

High Performance Graph Convolutional Networks with Applications in Testability Analysis Yuzhe Ma 1 , Haoxing Ren 2 , Brucek Khailany 2 , Harbinder Sikka 2 , Lijuan Luo 2 , Karthikeyan Natarajan 2 , Bei Yu 1 1 The Chinese University of Hong Kong 2

532 views • 18 slides

Batched Sparse Matrix Multiplication for Accelerating Graph Convolutional Networks Yusuke Nagasaka

Batched Sparse Matrix Multiplication for Accelerating Graph Convolutional Networks Yusuke Nagasaka , Akira Nukada , Kojima Ryosuke , Satoshi Matsuoka , Tokyo Institute of Technology Kyoto University RIKEN Center for

624 views • 26 slides

Kipf, T., Welling, M.: Semi-Supervised Classification with Graph Convolutional Networks Radim

Kipf, T., Welling, M.: Semi-Supervised Classification with Graph Convolutional Networks Radim petlk Czech Technical University in Prague 2 Overview - Kipf and Welling - use first order approximation in Fourier-domain to obtain an efficient

664 views • 27 slides

Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Transfer Learning with Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural Networks A breakthough Convolutional Neural Networks VGG-16 example Layers of Convolutional filters Bottleneck

625 views • 23 slides

Semi-supervised Geolocation via Graph Convolutional Networks Afshin Rahimi, Trevor Cohn and Tim

Semi-supervised Geolocation via Graph Convolutional Networks Afshin Rahimi, Trevor Cohn and Tim Baldwin July 16, 2018 1 / 25 Location Lost in Translation impacts impacts Social Media Society 2 / 25 Applications: Public Health Monitoring

710 views • 25 slides

Graph Convolutional Networks (GCNs) Dimitris Papatheodorou Aalto University

Graph Convolutional Networks (GCNs) Dimitris Papatheodorou Aalto University dimitrispapatheodorou95@gmail.com May 21, 2019 Dimitris Papatheodorou (Aalto University) GCNs May 21, 2019 1 / 35 Overview Introduction 1 Problem Setting Graph

710 views • 68 slides

Constant Curvature Graph Convolutional Networks Gregor Gary Octavian Bachmann B ecigneul

Constant Curvature Graph Convolutional Networks Gregor Gary Octavian Bachmann B ecigneul Ganea ETH Z urich MIT MIT Overview Overview Embeddings of graphs into hyperbolic and spherical space and their products Overview

1.13k views • 76 slides

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN) A.k.a. CNN or ConvNet Adit Deshpande, A Beginner's Guide To Understanding Convolutional Neural Networks. Digital Images Input array: an images

1.42k views • 72 slides

15-780 Graduate Artificial Intelligence: Convolutional and recurrent networks J. Zico Kolter

15-780 Graduate Artificial Intelligence: Convolutional and recurrent networks J. Zico Kolter (this lecture) and Ariel Procaccia Carnegie Mellon University Spring 2017 1 Outline Convolutional neural networks Applications of convolutional

1.32k views • 32 slides

Uncertainty-based graph convolutional networks for organ segmentation refinement Roger D.

Computer Aided Medical Procedures Uncertainty-based graph convolutional networks for organ segmentation refinement Roger D. Soberanis-Mukul 1 , Nassir Navab 1,2 , Shadi Albarqouni 1,3 1 Computer Aided Medical Procedures, Technische Universitt

751 views • 34 slides