Layer norm for batched graph

Questions
#1

Hi! I want to do layer_norm for batched graph, in other word, to do norm for subgraph of batched graph.
Do you have any ideas about how to use dgl to implement this?
Thank you!

#2

Firstly, layernorm is performed at channel level so I don’t think the graph size matters.
If you plan to perform some sort of normalization where graph size matters (e.g. batchnorm), I think our readout module would help (where you could take the sum/mean/max/… and broadcast a single feature to nodes/edges for each individual graph in a batched graph).

#3

Yes, I just want to know how to make the readout feature broadcast back to all nodes in batched graph

#4

Do you have an idea about how to do broadcast?

#5

We have provided broadcast_nodes and broadcast_edges in readout module.

1 Like
#6

Oh, thank you. It really helps!

#7

btw, feel free to contribute to dgl if you have finished a normalization layer, I think that would benefit many tasks if it works.