Some questions about the code of GCN

AllenWu18 · March 12, 2019, 2:13am

Here I researched the code of GCN in dal-master/examples/pytorch/gcn/train.py
and I don’t know where the code did the degree normalization in the paper
I have seen the variable declared:
degs = g.in_degrees().float()
norm = torch.pow(degs, -0.5)
norm[torch.isinf(norm)] = 0
if cuda:
norm = norm.cuda()
g.ndata[‘norm’] = norm.unsqueeze(1)
but I think the data “g.ndata[‘norm’]” hasn’t been used in the following code…
So my question is: Where does this variable work?Or is the normalization process has been packaged in the function dgl.nn.pytorch.GraphConv() already??

VoVAllen · March 12, 2019, 6:39am

It is used in the GraphConv module or other GCN implementations.

For example, here

github.com

dmlc/dgl/blob/master/examples/pytorch/gcn/gcn_spmv.py#L46


    stdv = 1. / math.sqrt(self.weight.size(1))
    self.weight.data.uniform_(-stdv, stdv)
    if self.bias is not None:
        self.bias.data.uniform_(-stdv, stdv)


def forward(self, h):
    if self.dropout:
        h = self.dropout(h)
    h = torch.mm(h, self.weight)
    # normalization by square root of src degree
    h = h * self.g.ndata['norm']
    self.g.ndata['h'] = h
    self.g.update_all(fn.copy_src(src='h', out='m'),
                      fn.sum(msg='m', out='h'))
    h = self.g.ndata.pop('h')
    # normalization by square root of dst degree
    h = h * self.g.ndata['norm']
    # bias
    if self.bias is not None:
        h = h + self.bias
    if self.activation:

AllenWu18 · March 12, 2019, 2:04pm

So is the meaning of it like follow:
the GCNLayer Class in code gcn_spmv is same to the function GraphConv in dgl.nn.pytorch?

minjie · March 12, 2019, 5:27pm

Hi, GraphConv module computes normalizer on-the-fly because one of the arugment is a graph object. This is useful when the graph structure is dynamic. So the codes you showed is not used in GraphConv module. They should be moved to gcn_spmv.py. Thanks for pointing this out!

AllenWu18 · March 14, 2019, 12:25pm

OK sorry for my late reply.
So what I mean is: if I just use the GraphConv(), should I use the above codes to declare and compute the normalization value?
I think it maybe compute in the function GraphConv() automatically?
Is my understanding correct??