As shown in the above figure, in your implementation of TreeLSTM, the accuracy could achieve up to 70% on fine-grained SST dataset, which largely surpasses the current reported results of 51%.
I am confused about why this happened, because the label of data is indeed the degree from 0 to 4, and you also said that the accuracy of your code is 51.76%.
So, could you tell me what’s wrong in my understanding of the printed “Acc”, and how to get the expected reported 51.76%?
On SST-5, people report “Root Acc” as the final accuracy.
But it seems your result (39.37) is too low, the result in our example (51.72) is based on DGL 0.1, so please allow me to check if I need to update code according to DGL 0.3 (where we update the kernels).
Hi, I have tried the tree-lstm code in our examples and I found it works fine with DGL 0.3.
To reproduce the result, make sure you have downloaded GloVe embedding as we described in README file (We would make it automatically later on).
Also, are you using the default setting? According to my experience, Tree-LSTM does not perform well with large batch size, by default the batch size is set to 25.
OK, thank you for your help, the problem may caused by the smaller hidden dimension, which is set to be 64 rather than the default 256.
