Interpretability of attention coefficients in a GAT?

Models & Apps
#1

I’m working on a GAT model for regression. Can the attention coefficients be used to identify influential nodes as it relates to the target variable? More generally, how might I interpret the attention coefficients for a GAT model for regression?

#2

It is still an open question whether attention can be used for model explanation, like the papers Attention is not Explanation and Attention is not not Explanation. However, it still characterizes the behavior of the model to some extents. Check out our blog for the discussion.

For GNN model explainability, perhaps you could check out the GNNExplainer module.

More generally, how might I interpret the attention coefficients for a GAT model for regression?

I don’t understand this question. Could you elaborate it more?

1 Like
#3

Thank you for the information! The articles and blog post were very helpful.

For the second question about attention coefficients for a GAT regression model, I was wondering if the attention coefficients would be dependent on the target feature. In other words, would the attention coefficients differ substantively if an alternative target was chosen?

#4

Yes, I think they will be different. Imagine predicting the subject of a paper will likely depend on the subjects of its cited papers while predicting the popularity of a paper will likely need author’s information.

#5

Great, thank you! I assumed that would be the case and just wanted to double check my assumption.

1 Like