Question about test/validation in GAT PPI Example

Questions
#1

I’m trying to create an inductive learning model for my own data (separate graphs), and added a confusion matrix to the evaluate() function.
When summing the confusion matrices of all batches, I noticed that the total sum of the matrix is equal to the sum of the square of the number of nodes in the graphs, rather than just the number of graphs.

This is because the evaluate function tests the predictions for the whole graph for each node, rather than just for the ‘subject node’. Therefore, each node is effectively evaluated as many times as there are nodes in its graph.

Why is the validation set up in this way? For inductive node classification, I’d expect that only the label of the ‘subject node’ is relevant, because all other labels in the graph were passed as input?

As a second question, I would like to train the network using only the classification of the other nodes as features. I would attempt to do this by just passing of the the labels other nodes as features. Is this a correct approach? How do ensure the label for the ‘subject node’ is not included?

#2

I might have not fully understood your question. By convention, PPI is used for node classification and we use 20 graphs for training, 2 for validation and 2 for test. With graph neural networks, we feed node features for input and predict node labels. What you suggest here sounds more like label propagation in probabilistic graphical models.

#3

On the test/validation question:
Looking at lines 38-40 and lines 107-111 of train_ppi.py, if my validation set contains one graph with 5 nodes and one graph with 10 nodes, the mean loss will be calculated over 5x5 + 10x10 = 125 comparisons of label and prediction. This is because for all 15 nodes total, every time the whole label/predict vector of the graph is used for the f1_score in evaluate().
I would expect the value to be calculated over 5+10=15 values, using each node only once; a bit more similar to what is described here. Unfortunately the referenced repository is no longer available.

On the label propagation:
I was under the impression that GNN’s would be able to perform without features other than the labels,
E.g. for GraphSAGE:
“<train_prefix>-feats.npy [optional] — A numpy-stored array of node features; ordering given by id_map.json. Can be omitted and only identity features will be used.” (my emphasis).
Could this work with GAT as well? Or does ‘identity features’ not mean what I thought?

#4

Maybe I’ve missed something. Currently we are using binary cross entropy for loss function. Both labels and predictions are of shape (N, D), where N is the number of nodes in a batch and D is the number of binary labels. Maybe you’ve modified the code with a different loss function?

By ‘identify features’, I think they are referring to the case where we do not have initial node features and want to learn them purely from supervision signals. For example, we can learn from scratch about user representations in the scenario of recommendation systems. But they are different from labels. It might be possible to do label propagation or learning node features from scratch with GAT, but I’ve not tried it before.