Questions about BatchNode mailbox

Questions
#1

Hi,

First, of all, thanks so much for this amazing graph computing framework!! Can I ask questions about the mailbox for BatchNode?

In our experiment, we have a batch of graphs with different structures. We build a python list to contain those graphs, then construct BatchGraph using dgl_bg = dgl.batch(batched_graphs), and feed it into our GCN model.

Here is the printed result of input graph feature(each line is a node representation), edges.src['feat'](source node representation of each edge) and mailbox information of each nodes. The printing code defined in gcn_msg and gcn_reduce function.

gnn_error
gnn_error1340×800 112 KB
.

According to my understanding, each line in edges.src['feat'] should be the same as the corresponding line in mailbox. However, here, they are different from the printed result.

In addition, all my mailbox lines do not contain the source node representation. To prove this, I used one of the source node representations in function gcn_msg, to test whether it exists in the the mailbox, but from the result(at the bottom in the following picture),

graph_error2
graph_error21056×795 96.7 KB

We can see that it is not contained in any mailbox row. So I am wondering why dgl just generates a random mailbox for me, instead of using the provided edge.src['feat']? Another question is why the shape of mailbox is torch.Size([148, 1, 50]) instead of torch.Size([148, 50])?

The following is my GCN related code

gnn_code_1
gnn_code_11115×655 99.5 KB

gcn_code_2
gcn_code_21012×440 63.7 KB

gcn_code_3
gcn_code_31003×427 58.2 KB

#2

Actually this is not true. The mailbox is the aggregation of all the incoming features, which depends on who is the node neighboors, not same for all nodes. You can refer to this and the figure below.

1535×536 34.2 KB

#3

Thanks so much for the such a important reference @VoVAllen ! Yeah, I found that I was totally wrong before. So if I want to implement the this function $$h_{u} = f((\sum h_{v})+h{u})$$, where v is all the neighbors of u, I need to use the message function return {'m': edges.src['feat'] + edges.dst['feat']} and reduce function fn.sum(msg='m', out='feat'), rather than my original one.

#4

the message function return {'m': edges.src['feat'] + edges.dst['feat']} and reduce function fn.sum(msg='m', out='feat')

if you use this, you would get $h{u}$ get added for several times for each edges. If you want to add the neighborhood and the node itself, you could add self loop on each node. Another solution is to use extra reduce function, which add neighborhood’s information and node feature in it.
Such as, msg func:{'m': edges.data['m']} and reduce func:{'feat': th.sum(nodes.mailbox['m'])+nodes.data['feat']}

#5

Hi, VoVAllen,

Thanks so much for the suggestion code. Yes, you are right. I find I did not understand that figure you provided me accurately before. So since there are two arrows from edge.dst.data, one through <apply>: node and another through <reduce>: node, it will be added multiple times. I hope this time I get it correct.

Thanks again!!

#6

Actually the reason of why it is added multiple times is only the usage of reduce function.

msg func: {'m': edges.src['feat'] + edges.dst['feat']}
This can be considered as following:

For each edges in the graph:
edges.data[‘m’] = edges.src[‘feat’] + edges.dst[‘feat’]
Now you add the destination node feature to the every edge’s m which it is connected by.

reduce func: fn.sum(msg='m', out='feat')
This can be considered as following:

For all nodes:
nodes.data[‘feat’] = sum(nodes.mailbox[‘m’])

And nodes.mailbox[‘m’] is the concat of all the edges’s m which it is connected by. You can see if one node has D edges, its feature will be added D times.

#7

Yes, you are totally correct!! Sorry, previously, I just considered the graph in my experiment which only has one edge for one node. I got it! Thanks so much for your series of patient and detailed illustration!