Tree LSTM source code

Questions
#1

Hi,
I read the tutorial code of tree-lstm. While I get confused if the code keeps consistent with the original formular.
g.ndata['iou'] = self.cell.W_iou((embeds)) * batch.mask.float().unsqueeze(-1) is called outside, but the value seems not be passed to the message passing, the iou = nodes.data['iou'] + self.b_iou only contains U_iou(h_cat) + b_iou, where is W_iou(x_z)?

I also have another question. In self.U_f = nn.Linear(2 * [h_size], 2 * [h_size]) why the output dimension is not 3 * [h_size]), and what is the dimension of nodes.mailbox['h'].size()? It seems the dimension of the element-wise multiplication for the following is not same.
f = th.sigmoid(self.U_f(h_cat)).view(*nodes.mailbox['h'].size())
c = th.sum(f * nodes.mailbox['c'], 1)

Thanks for your help.

The tutorial link:
https://docs.dgl.ai/en/0.6.x/tutorials/models/2_small_graph/3_tree-lstm.html

#2

The W_iou(x_z) are initializing the leaves. During prop_nodes, DGL will first perform message passing on the leaves as topological_nodes_generator gives. Those nodes do not have incoming edges, so message_func and reduce_func are skipped, so h and c are initialized from ndata['iou'] = self.cell.W_iou(embeds). After that, message passing runs on the internal nodes which only receives h and c from the children. W_iou are not used for these nodes.

The dimensionality of nodes.mailbox['h'].size() will be (num_nodes, num_neighbors, h_size). In our tutorial, the tree is always binary, and the left and right children will have different weight matrices (i.e. the summation in Eq 1-4). We implement that by concatenating the left child’s representation with the right child’s one, which is equivalent to reshaping. The result should be correct.

2 Likes
#3

Dear BarclayII,

Thanks so much for your kindly reply. Your post completely solved my questions.

#4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.