About CrossEntropy implemented in ogbn-arxiv example code

Questions
#1

In dgl/examples/pytorch/ogb/ogbn-arxiv/gcn.py, cross entropy is modified as

epsilon = 1 - math.log(2)

def cross_entropy(x, labels):
    y = F.cross_entropy(x, labels[:, 0], reduction="none")
    y = torch.log(epsilon + y) - math.log(epsilon)
    return torch.mean(y)

what does this function do?

#2

It’s a custom loss function. The function is copied from other code, I made a little changes but forgot to change the name.

#3

Could you please tell me the resource code/paper of this loss function?
And what do you think actually make this function work?

#4

I don’t know if anyone has used this loss function. In fact, this is my first time I tried this loss function, and it works on this dataset.
This loss function has the largest gradient for samples near the decision boundary, that is, when e^{x_{label}}=\frac{1}{2}\sum e^{x_i} where x is logits.
I guess it may be helpful to maximize accuracy.

1 Like
#5

Thanks for your reply