Hello,
I tried to use the DGL-LifeSci library for deep learning on protein structures but the performance was poor. I was advised to train the model on a bigger dataset and keep the convolution filters of the model to fine-tune the model on my classification task. Are there any examples I can follow to do this?
Basically, you need to train a model from scratch on a large dataset and then save the learned weights with
torch.save(model.state_dict(), PATH)
Then you can use the saved weights to initialize the model to train on the smaller dataset.
# Initialize a model instance
model = ...
# Load trained weights
model.load_state_dict(torch.load(PATH))
Wouldn’t i need to remove some layers because the larger dataset will have foreign samples which when trained would result in weights being changed in ways which i do not want?
What do you mean by foreign samples?
I mean these graphs would not belong to either of the classes I want to train my model on.
In that case, you can remove the last several layers and re-initialize them.
Is there any guide on how I can do so?
You may find this PyTorch discussion thread helpful: https://discuss.pytorch.org/t/how-to-load-part-of-pre-trained-model/1113/3