Hi everyone, I need guidance regarding Graph Neural Networks (GNNs). One example that I’ve struggled to understand well involves working with abstract graphs in our training dataset. Specifically, I aim to predict the degree of every node. Ideally, when applying graph regression, it should provide me with the average degree of the graph. While training such a model with low loss is straightforward, the challenge lies in its generalization capability to graphs of different sizes or distributions.

The setup I’m using involves GNN layers (various types like GIN, GCN, SAGE, etc.), adding self-loops, and incorporating dummy features at the node level (e.g., degree). However, I’ve encountered an issue where the model fails to learn relevant information; instead, it simply averages the node degrees regardless of the graph’s structure.

I initially believed that one GNN layer should suffice to capture neighborhood information. For aggregation, I’ve experimented with summation, averaging, and even max aggregation, yet none have resolved the problem. Currently, I use summation pooling layers alongside MSE loss.

What do you think could be the solution to this problem? Is it possible to generalize beyond the distribution of the training data, or is the issue rooted in the model learning a mapping that minimizes loss without capturing the correct relationships? I seek guidance on how to address this challenge and improve my understanding of GNN capabilities. If you know of any relevant papers addressing this problem, please share them with me so I can identify where I might be going wrong.