So i read the tutorial in DGL website for graph classification, and i was wondering is there any reason that there is no activation function being applied to the output of GraphConv layers? also what activation function do you guys suggest for me to use for the task of graph classification?
should i just use something like :
torch.relu(GraphConv(hidden_dim1, hidden_dim2 ))
or…?
Whether to apply an activation function to the output of GraphConv layers is rather a design choice as long as you do not apply an activation function to the final prediction of the whole model. In most cases ReLU will be the go-to option for activation functions.
What is the reason that i shouldnt add an activation function for the final prediction of model? is it becase CrossEntropyLoss applies a activation function itself or is there another reason?
For example, if you apply a ReLU layer to the final prediction of your model. Then there’s chance that the prediction will become zero and there will be a gradient vanishing issue in BP. This is generally undesired.