Geometric based manifold and graph representation learning and beyond

Abstract

Representation learning has shown great performances in various types of data, such as the data sampled with underlying manifold and graph domains. Specifically, in the field of manifold learning (ManL), one may prefer to generate a high quality data representation in the manifold latent space with the dimension same as the often assumed intrinsic dimension of the manifold, in the meanwhile, preserving some geometric properties of the manifold such as distance and angle. On the other hand, in terms of graph representation learning, the geometry/topology of the graph also plays a significant role in enhancing the learning accuracy of the popular graph neural networks (GNNs). In this thesis, our main focus is on how the underlying geometric information of the input data can assist existing manifold/graph learning models to generate more supreme learning outcomes. We start this thesis by presenting an ML algorithm in Chapter 2, namely Fixed Point Laplacian Mapping (FPLM) which guarantees to produce a bijective mapping (one to one and onto) between the input manifold data and their latent representations. FPLM prevents the potential information loss via classic ML learning methods and can be proven to produce a so-called chart map which generates a homeomorphism between input and output representations. We then turn our attention to the graph-structured data to further explore how graph geometric information can enhance the graph learning models (i.e., GNNs) in Chapter 3 and 4. In Chapter 5, we attempt to reduce the computational cost in GNNs by leveraging the graph knowledge distillation technique which decomposes GNN model into MLP based models without losing or even with higher learning accuracy. Lastly, in Chapter 6 we back to the learning theory of one of our previous works, namely p-Laplacian enhanced graph framelet model (pL-UFG) to thoroughly show its convergence, energy dynamic, dynamic interactions and relation to the non-linear diffusion. Furthermore, based on the results from our analysis of pL-UFG’s energy dynamic, we further simplified pL-UFG by adjusting its dominant learning domains, thus making pL-UFG adopt both homophilic and heterophilic graphs. Our theoretical work paves the path of evaluating GNNs via multiple facets.

Date of Thesis submission	2023
Original language	English
Awarding Institution	Western Sydney University
Supervisor	Yi Guo (Supervisor) & Junbin Gao (Supervisor)