Usage of GroupRevRes on model

[Noorulain]

I am applying GroupReveRes on this below mentioned model. But, it give me error every time.

Error is
"
RuntimeError: The size of tensor a must match the size of tensor b at non-singleton dimension 1"

class GNNLayer(nn.Module):
def init(self, feats, dropout=0.2):
super(GNNLayer, self).init()
# Use BatchNorm and dropout to prevent gradient vanishing
# In particular if you use a large number of GNN layers
self.norm = nn.BatchNorm1d(feats)
self.conv1 = GraphConv(feats, feats)
self.conv2 = GraphConv(feat, 30)
self.dropout = nn.Dropout(dropout)

def forward(self, g, x):
    x = self.norm(x)
    x = self.dropout(x)
    x1 = self.conv1(g,x)
    x2= self.conv2(g,x1)
    return x2

num_nodes = 5
num_edges = 20
feats = 32
groups = 2
g = dgl.rand_graph(num_nodes, num_edges)
x = torch.randn(num_nodes, feats)
conv = GNNLayer(feats // groups)
model = GroupRevRes(conv, groups)
out = model(g, x)

I have followed this whole code from dgl example “GroupRevRes — DGL 1.1 documentation”

Does anyone tell me, how can I apply GraphRevRes on the above mentioned model? Secondly, how I can select this group attribute? Here in above example, 2 value is chosen for group.

[mufeili]

Could you provide a minimal runnable script to reproduce the issue?

[Noorulain]

I have used this below code:

import torch

import dgl

x1 = torch.randn(128,128)

adj1 = torch.randn(128,128)

edge_index1 = adj1.nonzero().t()

row, col = edge_index1

edge_weight1 = adj1[row, col]

g1 = dgl.graph(

(torch.LongTensor(edge_index1[0]), torch.LongTensor(edge_index1[1]))

)

Assign a 3-dimensional node feature vector for each node.

g1.ndata[“x1”] = x1
g1.edata[“e1”] = edge_weight1

GCN as an example

from dgl.nn import GraphConv

import torch.nn as nn

import torch.nn.functional as F

import torch

class GCN(nn.Module):

def __init__(self, in_feats, h_feats, num_classes):

    super(GCN, self).__init__()

    self.conv1_1 = GraphConv(in_feats, h_feats)

    self.conv2_1 = torch.nn.Linear(h_feats, num_classes)

def forward(self, g1, in_feat1, e_weight1):

    h1_1 = self.conv1_1(g1, in_feat1, e_weight1)

    h1 = F.relu(h1)

    h = self.conv2_1(h1)

    return h

groups = 2

#features shape

feats = x1.shape[1]

from dgl.nn import GraphConv, GroupRevRes

conv = GCN((feats // groups), 16, 10)

model = GroupRevRes(conv, groups)

out = model(g1, x1, g1.edata[“e1”])

The error which I have got is mentioned below:

---

DGLError Traceback (most recent call last)
in <cell line: 7>()
5 conv = GCN((feats // groups), 16, 10)
6 model = GroupRevRes(conv, groups)
----> 7 out = model(g1, x1, g1.edata[“e1”])

8 frames
/usr/local/lib/python3.10/dist-packages/dgl/nn/pytorch/conv/graphconv.py in forward(self, graph, feat, weight, edge_weight)
437 if weight is not None:
438 if self.weight is not None:
→ 439 raise DGLError(
440 “External weight is provided while at the same time the”
441 " module has defined its own weight parameter. Please"

DGLError: External weight is provided while at the same time the module has defined its own weight parameter. Please create the module with flag weight=False.

[mufeili]

The error has nothing to do with GroupRevRes. You need to change h1_1 = self.conv1_1(g1, in_feat1, e_weight1) to h1_1 = self.conv1_1(g1, in_feat1, edge_weight=e_weight1). Even in that case, GroupRevRes currently does not support edge weights.

[Noorulain]

@mufeili ,Do you have an idea, when will we able to use GroupRevRes along with edge weight support? The second limitation which is written on its documentation, input and output node representation should be same, it means, we cannot use this model on
in_feats = 32
h_feats = 8
num_classes = 4
self.conv1 = GraphConv(in_feats, h_feats)
self.conv2 = torch.nn.Linear(h_feats, num_classes)

[mufeili]

Regarding the support for edge weights, the main issue lies at dgl/python/dgl/nn/pytorch/conv/grouprevres.py at master · dmlc/dgl · GitHub and dgl/python/dgl/nn/pytorch/conv/grouprevres.py at master · dmlc/dgl · GitHub. Currently, we chunk all arguments. However, we can actually check if the first dimension matches the number of nodes. If not, then chunking should not be performed. It will be great if you can help contribute that.

Regarding the second issue, you can use GroupRevRes for intermediate graph convolutional layers only.

[Noorulain]

@mufeili , I have the multilabel binary classification task, I want to improve the performance of F1 score from 0.30 to 0.37 or 0.38. With DGL GSAGE with 3 layers, I have got the score of F1 of 0.30. I have applied GraphSAGE model along with edge weight support using the DGL library. I have made a Deep model of GSAGE where I have created 3 layers, after every GSAGE layers I have used the activation function relu by seeing the DGL documentation. Till the addition of 3 layers of GSAGE model, I have seen an improvement in performance in F1 score, but, when I add the GSAGE layer 4th time or 5th time, my performance in F1 score decrease. Can you give any suggestion, about improving the performance using GSAGE model after the addition of 4 layer to remove the overfitting? Can I add anything to improve the performance by addition of more layers in GSAGE which are supported in current DGL along with edge weight support?

[mufeili]

More layers of message passing do not necessarily help. You may try some models other than GraphSAGE and see if they yield a better performance. Also for three layers, you may not need GroupRevRes.

[Noorulain]

@mufeili , can you suggest some other models with edge weight support of DGL, which should I try except of GCN. I have tried GAT and GIN. But, both performances are lower than GSAGE.

[mufeili]

It’s hard to say. You may want to check some papers dealing with data similar to yours.