Unique Case of Heterogeneous Graph: Optimizing Representation and Training Strategies

[walidgeuttala]

Hello,

I have a heterogeneous graph containing 5 different types of edges and 1 node type. Initially, there are 5 different types of edges in the graph, most commonly appearing as symmetric shapes. Additionally, there are no nodes with a degree of 0 in any of these 5 types of edges.

What is the best solution to address this problem? Should I treat the edges as a homogeneous graph, assigning a unique ID to each type, and then perform message passing using node and edge IDs? Alternatively, should I utilize the heterogeneous graph approach, even though it requires duplicating node features for each type of edge? On the other hand, there’s the possibility of training different GNN layers separately for each type of edge and then combining them in the final layer using an MLP. Or, should I use the same GNN layer for all different types of edges?

I hope you can provide guidance and share your thoughts on the optimal representation of the graph and for training the DGL model.

[walidgeuttala]

Rhett-Ying

[minjie]

@walidgeuttala

Should I treat the edges as a homogeneous graph, assigning a unique ID to each type, and then perform message passing using node and edge IDs? Alternatively, should I utilize the heterogeneous graph approach, even though it requires duplicating node features for each type of edge?

Both approaches are fine. I will start with heterogeneous GNNs. Note that DGL heterogeneous graph supports your case natively without duplicating node features. An extreme version of your case is knowledge graph where there is only one node type (entity) but thousands or hundreds of thousands of edge types (relations), and DGL works fine.

@walidgeuttala

On the other hand, there’s the possibility of training different GNN layers separately for each type of edge and then combining them in the final layer using an MLP. Or, should I use the same GNN layer for all different types of edges?

DGL’s HeteroGraphConv trains different graph convolutions (same architecture but different parameterization) for different edge type. You can then combine the output using MLP. Check out our heterogeneous GNN example for references.

[walidgeuttala]

Thank you, I fixed the problem. When I have a heterogeneous graph with only one type of node, the ndata attribute of the graph forces me to use the tensor shape instead of a dictionary. However, when I feed the ndata into the HeteroGraphConv, it must be a dictionary even if it has only one node type.

On the other hand, when I have n types of edges and 1 type of node, how should the last hidden presentation be fed to the next layer when using HeteroGraphConv ? Since GNNs update the representation of the same nodes, there may be some considerations to address

[Rhett-Ying]

Could you refer to HeteroGraphConv — DGL 2.1.0 documentation for the usage of HeteroGraphConv?

import dgl
import torch

g = dgl.heterograph({
    ('user', 'follows', 'user'): ([0, 1, 2], [1, 2, 0]),
    ('user', 'plays', 'user'): ([0, 1, 2], [0, 1, 2]),
    ('user', 'watches', 'user'): ([0, 1, 2], [1, 2, 1])
})
g.nodes['user'].data['h'] = torch.randn(3, 5)
g.edges['follows'].data['h'] = torch.randn(3, 5)
g.edges['plays'].data['h'] = torch.randn(3, 5)
g.edges['watches'].data['h'] = torch.randn(3, 5)

import dgl.nn.pytorch as dglnn
conv = dglnn.HeteroGraphConv({
    'follows': dglnn.GraphConv(5, 5),
    'plays': dglnn.GraphConv(5, 5),
    'watches': dglnn.GraphConv(5, 5)
})
h_dict = conv(g, {'user': g.nodes['user'].data['h']})
print(h_dict)

Refer to this example for multiple layers: dgl/examples/pytorch/rgat/train.py at 658b2086b09bbd76c3d3f488af2b155a1c921052 · dmlc/dgl · GitHub

[walidgeuttala]

Thank you, it solved the problem.