Query regarding Fine-Tuning/Feature Extraction


I tried to use the DGL-LifeSci library for deep learning on protein structures but the performance was poor. I was advised to train the model on a bigger dataset and keep the convolution filters of the model to fine-tune the model on my classification task. Are there any examples I can follow to do this?

Basically, you need to train a model from scratch on a large dataset and then save the learned weights with

torch.save(model.state_dict(), PATH)

Then you can use the saved weights to initialize the model to train on the smaller dataset.

# Initialize a model instance
model = ...
# Load trained weights

Wouldn’t i need to remove some layers because the larger dataset will have foreign samples which when trained would result in weights being changed in ways which i do not want?

What do you mean by foreign samples?

I mean these graphs would not belong to either of the classes I want to train my model on.

In that case, you can remove the last several layers and re-initialize them.

Is there any guide on how I can do so?

You may find this PyTorch discussion thread helpful: https://discuss.pytorch.org/t/how-to-load-part-of-pre-trained-model/1113/3