Accuracy of training beyond 1.0

Dear all,

Have you experienced training a binary classification that produces the accuracy > 1.0? Before, the training throw some warnings, but I couldn’t manage to investigate it properly. I guess it perhaps because of the type of feature I defined, which is torch.float32. But I looked at https://discuss.dgl.ai/t/builtin-functions-give-unsupported-dtype-error-on-torch-double-tensors/411/2, it is suggested to use this data type. So, what could be the cause of this result?

Thanks

/usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py:3335: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/usr/local/lib/python3.6/dist-packages/numpy/core/_methods.py:161: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
Epoch 00000 | Time(s) nan | Loss 0.6978 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00001 | Time(s) nan | Loss 0.6924 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00002 | Time(s) nan | Loss 0.6918 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00003 | Time(s) nan | Loss 0.7002 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00004 | Time(s) nan | Loss 0.6934 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00005 | Time(s) nan | Loss 0.6924 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00006 | Time(s) nan | Loss 0.6930 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00007 | Time(s) nan | Loss 0.6937 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00008 | Time(s) nan | Loss 0.6929 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00009 | Time(s) nan | Loss 0.6968 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00010 | Time(s) nan | Loss 0.6954 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00011 | Time(s) nan | Loss 0.6925 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00012 | Time(s) nan | Loss 0.6932 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00013 | Time(s) nan | Loss 0.6931 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00014 | Time(s) nan | Loss 0.6931 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00015 | Time(s) nan | Loss 0.6931 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00016 | Time(s) nan | Loss 0.6931 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00017 | Time(s) nan | Loss 0.6931 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00018 | Time(s) nan | Loss 0.6931 | Accuracy 241.0000 | ETputs(KTEPS) nan
Epoch 00019 | Time(s) nan | Loss 0.6931 | Accuracy 241.0000 | ETputs(KTEPS) nan

Hi,

Could you tell us more about which code are you running? The problem doesn’t seem due to DGL

Thanks @VoVAllen,

please find the codes I use:

class GNNLayer2(nn.Module):
  def __init__(self, ndim_in, edims, ndim_out, activation):
    super(GNNLayer2, self).__init__()
    self.W_msg = nn.Linear(ndim_in + edims, ndim_out)
    self.W_apply = nn.Linear(ndim_in + ndim_out, ndim_out)
    self.activation = activation

  def message_func(self, edges):
    return {'m': F.relu(self.W_msg(torch.cat([edges.src['h'], edges.data['h']], 1)))}  
    
  def forward(self, g_dgl, nfeats, efeats):
    with g_dgl.local_scope():
      g = g_dgl
      g.ndata['h'] = nfeats
      g.edata['h'] = efeats
      g.update_all(self.message_func, fn.sum('m', 'h_neigh'))
      g.ndata['h'] = F.relu(self.W_apply(torch.cat([g.ndata['h'], g.ndata['h_neigh']], 1)))
      return g.ndata['h']

class NetArchie2(nn.Module):
  def __init__(self, ndim_in, ndim_out, edim, activation, dropout):
    super(NetArchie2, self).__init__()
    self.layers = nn.ModuleList()
    self.layers.append(GNNLayer2(ndim_in, edim, 50, activation))
    self.layers.append(GNNLayer2(50, edim, 25, activation))
    self.layers.append(GNNLayer2(25, edim, ndim_out, activation))
    self.dropout = nn.Dropout(p=dropout)    

  def forward(self, g_dgl, nfeats, efeats):
    g = g_dgl
    for i, layer in enumerate(self.layers):
        if i != 0:
            nfeats = self.dropout(nfeats)
        nfeats = layer(g, nfeats, efeats)
    return nfeats

def evaluate(model, g, nfeats, efeats, labels, mask):
    model.eval()
    with torch.no_grad():
        logits = model(g, nfeats, efeats)
        logits = logits[mask]
        labels = labels[mask]
        _, indices = torch.max(logits, dim=1)
        correct = torch.sum(indices == labels)
    return correct.item() * 1.0 / len(labels)    

optimizer = torch.optim.Adam(net.parameters(), lr=1e-2)
loss_fcn = torch.nn.BCEWithLogitsLoss()
net = NetArchie2(5, 1, 2, F.relu, 0.5)

for epoch in range(100):
    net.train()
    logits = net(g, nfeats, efeats)
    loss = loss_fcn(logp[train_mask], labels[train_mask])
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    acc = evaluate(model, g, nfeats, efeats, labs, eval_mask)
    print("Epoch {:05d} | Time(s) {:.4f} | Loss {:.4f} | Accuracy {:.4f} | "
        "ETputs(KTEPS) {:.2f}". format(epoch, np.mean(dur), loss.item(),
                                        acc, g.number_of_edges() / np.mean(dur) / 1000))

The size of node features is torch.Size([3420, 5]), edge feature is torch.Size([12360, 2]), and target binary labels is torch.Size([3420, 1]).

I’m trying to perform binary classification that encounter nodes and edges features. Please advice me on how should I proceed with this task.

Thank you very much @VoVAllen.