Suppose each node in the graph is associate with an image, how can we encode the image during training? instead of encoding them when building the graph? I am thinking of something like this:
class model(nn.Module): def __init__(self): self.cnn = cnn() self.gcn = SAGEConv() def forward(self, nodeID) image = get_image(nodeID) image_feature = self.cnn(image) node_embedding = self.gcn(graph, image_feature)
Is this doable? If so, how should I define the data loader to generate the nodeID? Can I use the NodeDataLoader/EdgeDataLoader that dgl provides?
Thank you in advance!