Feeding the isolated nodes with features into GCN or GAT affect the final logits for classification tasks?

[xdwang0726]

Hi, I am wondering does feeding the isolated nodes with features into GCN and GAT affect the final outputs for classification tasks?

Those isolated nodes have original features and will they be changed after GCN or GAT? (I suppose that there is no message passing through isolated nodes, and their feature embeddings will not be changed after applied GCN or GAT.)

Also, It seems that GAT does not allow isolated nodes, and one option to deal with isolated nodes is to add self-loop. I am wondering does adding self-loop affect the final outputs for classification tasks?

Thanks!

[mufeili]

For an isolated node, i.e. a node without incoming edges, message passing will result in a zero output. In such cases, it’s common to add self loops or residual connections.

[xdwang0726]

Thanks for your reply! I am wondering if adding self-loop, will it change the original node features? (for my understanding, the node features should stay same after GAT or GCN if the node is isolated. btw, I am using nodes features for classification tasks.)

[mufeili]

The node feature will not stay the same if a node is isolated. Consider the matrix form AX where A is the adjacency matrix. If A is fully-zero, then the result of the multiplication will be fully-zero. Adding self-loops allows having non-zero output node representations and can be similar to skip connections.

[xdwang0726]

Thanks for your reply! Sorry, I didn’t quite understand why the original features for isolated nodes need to be changed after GCN or GAT. Maybe I didn’t state my concerns clearly, let me form my question in this way: if we are aiming for a node classification task, and we have some isolated nodes in the graph. In which is the correct way to proceed:

1. remove the isolated nodes in the graph, and use GCN/GAT to gain new representations for connected nodes, and then add the original representations of the isolated nodes to get the full node representations.
2. add self-loop to the isolated nodes and send the graph to GCN/GAT , and then get new full node representations.

Thanks!

[mufeili]

I will not say it’s the “correct” way, but people typically go with the second way. This is equivalent to apply an MLP to the representations of the isolated nodes.