If i am not mistaken, based on their paper the GIN with 1 layer MLP and with both poolings set to mean, should be very similar to GCN.
i am using the GIN code provided by dgl on my own dataset : https://github.com/dmlc/dgl/tree/master/examples/pytorch/gin
but the problem is that when i compare the GIN with mean/mean pooling and 1 layer MLP with my own GCN that i implemented using dgl’s GraphConv, it is so much worse!
for example this is GIN after 25 epoch :
train set - average loss: 2.4517, accuracy: 6.7742%
valid set - average loss: 2.4659, accuracy: 8.0460%
epoch-25:
and here’s with sum neighbors and sum graph pooling after 25 epoch :
valid set - average loss: 291.1245, accuracy: 16%
train set - average loss: 101.9377, accuracy: 16%
and this is my GCN :
train set - average loss: 1.5589, accuracy: 50.8065%
valid set - average loss: 1.5502, accuracy: 43.6782%
epoch-25:
everything is the same, the dataset, learning rate, optimization parameters, this is basically my GCN :
class Classifier(nn.Module):
def __init__(self, in_dim, hidden_dim1 , n_classes):
super(Classifier, self).__init__()
self.conv1 = GraphConv(in_dim, hidden_dim1)
self.classify = nn.Linear(hidden_dim1 , n_classes)
def forward(self, g , h ):
h = torch.relu(self.conv1(g, h))
g.ndata['h'] = h
hg = dgl.mean_nodes(g, 'h')
return self.classify(hg)
and my GIN code is basically using model = GIN(…), the same as the provided example in dgl. and the way i calculate loss and accuracy is exactly the same in both, loss function the same too CrossEntropyLoss, no dropout, my code is basically very similar to main.py of the GIN example.
is this normal? GIN just seems so unstable, the accuracy goes up and down very much, when i switch to sum sum the loss goes off the roof, everything seems to unstable and not good, is this normal? changing the mean/mean to anything else doesn’t help much either…
This is the code of net evaluation for both GCN and GIN, identical to provided examples in dgl :
def eval_net(args, net, dataloader, criterion):
net.eval()
total = 0
total_loss = 0
total_correct = 0
for data in dataloader:
graphs, labels = data
feat = graphs.in_degrees().view(-1, 1).float().to(args.device)
graphs = graphs.to(args.device)
labels = labels.to(args.device)
total += len(labels)
outputs = net(graphs, feat)
_, predicted = torch.max(outputs.data, 1)
total_correct += (predicted == labels.data).sum().item()
loss = criterion(outputs, labels)
total_loss += loss.item() * len(labels)
loss, acc = 1.0*total_loss / total, 1.0*total_correct / total
net.train()
return loss, acc
and my train function which again is identical in both of them :
def train(args, net, trainloader, optimizer, criterion, epoch):
net.train()
running_loss = 0
total_iters = len(trainloader)
bar = tqdm(range(total_iters), unit='batch', file=sys.stdout)
for pos, (graphs, labels) in zip(bar, trainloader):
labels = labels.to(args.device)
feat = graphs.in_degrees().view(-1, 1).float().to(args.device)
graphs = graphs.to(args.device)
outputs = net(graphs, feat)
loss = criterion(outputs, labels)
running_loss += loss.item()
optimizer.zero_grad()
loss.backward()
optimizer.step()
# report
bar.set_description('epoch-{}'.format(epoch))
bar.close()
# the final batch will be aligned
running_loss = running_loss / total_iters
return running_loss
higher epochs doesnt help either, GIN with 100 epochs is still worse than GCN with 20 epoch! why?!