FaLa: feature-augmented location-aware transformer network for graph representation learning

Data augmentation is a technique that improves the ability of neural networks to make accurate predictions by increasing the size of the training dataset. However, it is still uncertain how to properly use data augmentation on graph data in order to boost the performance of graph neural networks (GNNs). While the majority of current graph regularizers primarily concentrate on modifying the topological structures of graphs by adding or deleting nodes and edges, we propose a technique to augment node and edge features in order to improve efficiency. More precisely, we use cosine similarity to perform cross-functions on the existing features. This allows us to generate new graph features, such as node and edge attributes, as well as new graph structures. We then merge these new features together to provide specific pairs of inputs for graph transformer networks. We next propose feature-augmented location-aware transformer (FaLa) networks that generalize well across several graph learning problems by combining the positional and structural encoding of nodes. Additionally, in order to avoid collecting duplicate data from other features, we apply a disparity restriction during training on these latent node embeddings. Experimental findings across eleven real-world datasets demonstrate that our methodology enhances node classification accuracy by a significant margin of 2.8% to 22.7%, link prediction by 1.6% to 8.5%, and graph classification by 7.0% to 39.8% compared to the state-of-the-art graph convolutional network (GCN) and graph transformer (GT) models. This clearly shows the importance of augmenting graph features and location encoding for efficient graph representation learning.