Residual connection with neighbourhood sampling on GAT

[Krishna]

Hi,

I am trying to implement neighbourhood sampling technique with GATConv. I have followed the below implementation.

The above implementation is from the DGL GitHub examples for the GAT of dataset
ogbn-products. So, from the above snippet I understood that the output shape of the GATConv layer will not be the same as the input. How do I implement residual connections in this case?

As for the full batch training, I usually follow the norm of adding the input to the output of the GATConv layer.

[BarclayII]

You only need to account the residual connection on the output nodes, so you can do things like this:

h_dst = h[:block.num_dst_nodes()]
h_delta = layer(block, (h, h_dst))
h = h_dst + h_delta