The probability of a positive class in the binary classification

Questions
#1

Dear all,

How do we get the probability of the positive class and its class from the binary classification output?

I have a trained model (binary classification) and I want to validate the model with a new graph and expect the model to list the class of each node in the new graph with its probability.

Thanks in advance

#2

I assume there is only a scalar per datapoint for the model output? If this is the case, it depends on whether sigmoid has been applied. If sigmoid has been applied, then the output is already the probability for positive class, otherwise you can get that by applying the sigmoid activation function.

#3

Thanks @mufeili,

You have been so much helpful for me, and do apologize if I have shot you with plenty of questions. To confirm my understanding about this, I proceed with this implementation

trained_model = torch.load("./mod_gcn.pt")
inference = trained_model(g_infer, g_infer.ndata[“nodeFeat”], g_infer.edata[“edgeFeat”])
interference_prob= torch.sigmoid(inference)
—>tensor([[0.5556],[0.5381],[0.9999],[0.9905]], grad_fn=<SigmoidBackward>)

  1. The result of the sigmoid is the list of probabilities for all 4 nodes in the graph that its node I predict (correct me if I’m wrong). But how do I know which class (binary 0 or 1) that those probabilities are representing?
  2. And can you point to the example of early stopping for training the binary classification?

Thanks a lot

#4
  1. “But how do I know which class (binary 0 or 1) that those probabilities are representing?” This is decided by the use of the binary cross entropy loss function, in which case we always treat the model output as the logits (values before sigmoid) for the class 1.
  2. The script performs multi-label binary prediction with early stopping, which you may find helpful: https://github.com/awslabs/dgl-lifesci/blob/master/examples/property_prediction/csv_data_configuration/classification_train.py#L77
#5

Thank so much @mufeili,

In that case, I sometimes get 0.5000 as the outcome of the probability of having 1, which means 0.5 of having 0. How we normally deal with this result?

Thanks a lot

#6

If this is very common, I would recommend using ROC AUC or PRAUC as the evaluation metric instead.