R-GCN Example - Weight Generation and Input Layer Node Features

Questions
,
#1

Screenshot from 2020-10-15 21-29-44
Screenshot from 2020-10-15 21-29-44835×675 66.4 KB

In the given code at https://docs.dgl.ai/tutorials/models/1_gnn/4_rgcn.html I can’t really understand how the weights are created from the weight bases. Why do we change the shape first and use matmul then? What is the underlying idea?

And also in the message function for the input layer, why do we calculate the message as

embed = weight.view(-1, self.out_feat)
            index = edges.data['rel_type'] * self.in_feat + edges.src['id']
            return {'msg': embed[index] * edges.data['norm']}

? What is the underlying idea? Why aren’t there node features coming from the data?

I am really new to the subject so sorry if these are really trivial questions.

1 Like
#2

In the given code at https://docs.dgl.ai/tutorials/models/1_gnn/4_rgcn.html I can’t really understand how the weights are created from the weight bases. Why do we change the shape first and use matmul then? What is the underlying idea?

See 2.2 in the paper for a description of basis-decomposition.

And also in the message function for the input layer, why do we calculate the message as

embed = weight.view(-1, self.out_feat)
            index = edges.data['rel_type'] * self.in_feat + edges.src['id']
            return {'msg': embed[index] * edges.data['norm']}

? What is the underlying idea? Why aren’t there node features coming from the data?

This is because the dataset does not have initial node features and node embeddings are learned from scratch.

1 Like
#3

I understand that there is a basis-decomposition going on what how that codes implement that. About the 2nd part,
edges.data['rel_type'] * self.in_feat + edges.src['id']
why do we specifically choose this row for message? Is it just to get some random value?

#4

The tutorial assumes a setting where there are no initial node features and the model learns node embeddings from scratch. The tutorial defines the input layer as follows:

RGCNLayer(self.num_nodes, self.h_dim, self.num_rels, self.num_bases,
          activation=F.relu, is_input_layer=True)

I think this part may be improved for better readability.

In this case, self.in_feat is simply self.num_nodes, i.e. the number of nodes in each relation. Therefore the index here corresponds to the row in the embedding for the source node of the edge.

1 Like
#5

That makes sense now. Thanks a lot.