When Does Self-Supervision Help Graph Convolutional Networks? Yuning - PowerPoint PPT Presentation

When Does Self-Supervision Help Graph Convolutional Networks? Yuning You * , Tianlong Chen * , Zhangyang Wang, Yang Shen Texas A&M University * Equal Contribution Department of Electrical and Computer Engineering 1 This work was presented at ICML 2020

Contents Motivation • Contribution 1. How to incorporate self-supervision (SS) in graph • convolutional networks (GCNs)? Contribution 2. How to design SS tasks to improve model • generalizability? Contribution 3. Does SS boost model robustness? • Conclusions • Department of Electrical and Computer Engineering 2

Motivation Semi-supervised (SS) learning is an important field of • graph-based applications with abundant unlabeled data available; Using unlabeled data, SS is a promising technique in the few-shot • scenario for computer vision; SS in graph neural networks for graph-structured data is still • under-explored with an exception (M3S, AAAI’19). Department of Electrical and Computer Engineering 3

Contribution 1. How to incorporate SS in GCNs? We perform a systematic study on SS + GCNs: • – 1. How to incorporate SS in GCNs? Train Train in the with SS downstream tasks task • Pretraining & finetuning; Generate pseudo Train in the labels via SS downstream treated as true task • Self-training (M3S, AAAI’19); labels Repeat several rounds • Multi-task learning. Train in the downstream task together with SS tasks Department of Electrical and Computer Engineering 4

Contribution 1. How to incorporate SS in GCNs? Multi-task learning: Train in the downstream • task together with SS tasks – Empirically outperforms other two schemes; – We regard the SS task as a regularization term throughout the network training; – Act as a data-driven regularizer. Department of Electrical and Computer Engineering 5

Contribution 2. How to design SS tasks to improve generalizability? We investigate three SS tasks: • We illustrate that different SS tasks benefit generalizability in different • cases. Department of Electrical and Computer Engineering 6

Contribution 3. Does SS boost robustness? We generalize SS into adversarial training: • – Adversarial training: – SS + Adversarial training: Department of Electrical and Computer Engineering 7

Contribution 3. Does SS boost robustness? We show that SS also improves GCN robustness without requiring • larger models or additional data. – Clu is more effective against feature attacks; – Par is more effective against links attacks; – Strikingly, Comp significantly boosts robustness against link attacks and link & feature attacks on Cora. Department of Electrical and Computer Engineering 8

Conclusion We demonstrate the effectiveness of incorporating • self-supervised learning in GCNs through multi-task learning; We illustrate that appropriately designed multi-task • self-supervision tasks benefit GCN generalizability in different cases; We show that multi-task self-supervision also improves • robustness against attacks, without requiring larger models or additional data. Department of Electrical and Computer Engineering 9

Thank you for listening. Paper: https://arxiv.org/abs/2006.09136 Code: https://github.com/Shen-Lab/SS-GCNs Department of Electrical and Computer Engineering 10

When Does Self-Supervision Help Graph Convolutional Networks? Yuning - PowerPoint PPT Presentation

When Does Self-Supervision Help Graph Convolutional Networks? Yuning You * , Tianlong Chen * , Zhangyang Wang, Yang Shen Texas A&M University * Equal Contribution Department of Electrical and Computer Engineering 1 This work was presented

EMNLP 2017 Copenhagen Contributions } Syntactic Graph Convolutional Networks }

On the Experimental transferability of Spectral Graph Convolutional Networks Masters project

Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation Jiaxuan You*,

Convolutional Kernel Networks for Graph-Structured Data Dexiong Chen 1 Laurent Jacob 2 Julien

Graph Convolutional Networks

Simplifying Graph Convolutional Networks Amauri Holanda Felix Wu* Tianyi Zhang* Christopher

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

Graph Convolutional Network Heting Gao University of Illinois at Urbana-Champaign

Supervision Strengthening Our Practice The plan Supervision what is it? Benefits

DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation

L 2 -GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks Yuning You * ,

Extracting and Modeling Relations with Graph Convolutional Networks Ivan Titov with Diego

Graph Representation Learning with Graph Convolutional Networks Jure Leskovec Networks: Common

High Performance Graph Convolutional Networks with Applications in Testability Analysis Yuzhe Ma 1

MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing Sami

Batched Sparse Matrix Multiplication for Accelerating Graph Convolutional Networks Yusuke Nagasaka

Kipf, T., Welling, M.: Semi-Supervised Classification with Graph Convolutional Networks Radim

Affinity Graph Supervision for Visual Recognition Paper ID: 7437 Chu Wang 1 , Babak Samari 1 ,

Semi-supervised Geolocation via Graph Convolutional Networks Afshin Rahimi, Trevor Cohn and Tim

Graph Convolutional Networks (GCNs) Dimitris Papatheodorou Aalto University

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning Qimai Li, Zhichao

Guardian : Evaluating Trust in Online Social Networks with Graph Convolutional Networks Wanyu Lin

Supervision Mandatory Webinar 4 Webinar overview I. Background II. Why supervision? III.

Constant Curvature Graph Convolutional Networks Gregor Gary Octavian Bachmann B ecigneul