Join the Learning on Graphs and Geometry Reading Group: hannes-stark.com/logag-readin...
Paper “On Recoverability of Graph Neural Network Representation | Maxim Fisherman": arxiv.org/abs/2201.12843
Abstract: Despite their growing popularity, graph neural networks (GNNs) still have multiple unsolved problems, including finding more expressive aggregation methods, propagation of information to distant nodes, and training on large-scale graphs. Understanding and solving such problems require developing analytic tools and techniques. In this work, we propose the notion of recoverability, which is tightly related to information aggregation in GNNs, and based on this concept, develop the method for GNN embedding analysis. We define recoverability theoretically and propose a method for its efficient empirical estimation. We demonstrate, through extensive experimental results on various datasets and different GNN architectures, that estimated recoverability correlates with aggregation method expressivity and graph sparsification quality. Therefore, we believe that the proposed method could provide an essential tool for understanding the roots of the aforementioned problems, and potentially lead to a GNN design that overcomes them.
Authors: Maxim Fishman, Chaim Baskin, Evgenii Zheltonozhskii, Almog David, Ron Banner, Avi Mendelson
Twitter Hannes: / hannesstaerk
Twitter Dominique: / dom_beaini
Twitter Valence Discovery: / valence_ai
Reading Group Slack: join.slack.com/t/logag/shared...
~
Chapters
00:00 Intro & Agenda
01:36 - Motivation
11:56 - Measure Theory
20:59 - Embedding Expressiveness + Recovery Loss
35:55 - Aggregation Method vs. Recoverability
39:01 - Recoverability vs. Regularization
45:16 - Experimental Results and Discussion
53:28 - Recoverability vs. MI and Results
1:00:30 - Q+A
15 июл 2024
