Inference on unseen nodes in hetero graphs

[saqqa95]

I come from a non computer science background and i am new to the field of machine learning. I was trying Dynamic Graph Neural Networks for sequential recommendation for a link prediction task to get the next command prediction for a building information modelling software. However, i am not sure how this model can be generalized to new unseen nodes.

The graph consists of user and item nodes, and the edges denote the interaction of users with item. Using an order aware graph attention mechanism the node embeddings are updated. Predition takes place using a dot product between the final user and item node embeddings to get the scores. If a new node comes to the graph, and lets say, it interacts with 5 item nodes, how can we get an embedding to the new user without retraining the whole model again.

I tried using a simple union/intersection method and average the embeddings of similar users (that interacted with the same 5 items), and passed the embedding to the model, but it yielded nan metrics. Any guidance would really appreciated.

[minjie]

What you described is a typical inductive setting in graph machine learning, where model at training time does not have the full information of instances at serving time. Traditional embedding based models (e.g., network embedding) are transductive, but GNNs are designed to be inductive. In your case, the model, once trained, should be able to collect and aggregate neighborhood information into a proper node-level representation for final prediction. No model retraining is needed. Note that many GNN work (I didn’t check the paper you pasted) will combine node embedding with GNN which makes the model transductive again; something you probably want to avoid.

[saqqa95]

I tried adding some user nodes to the graph and run the inference using the pretrained model state dict, and initialized them by averaging the embeddings of similar old users, but it couldn’t generate predictions for the new user nodes. i am not sure if there is something wrong i did, or it is generally not able to generalize to unseen nodes.