Attentive fp implementation

Hi, I’ve used the attentive fp implement in dgl-lifesci for solubility prediction ( i.e. the ESOL dataset), but I can’t reproduce the result in the paper ( 0.50). I can only reach 0.54 after some hyperparameter tunning. What could be the most possible reason?
Any help is appreciated.

How was the dataset split? If it is a random split, then we will not be able to reproduce the model performance since we may not be able to reproduce the split.

Yeah, It’s a random split