Efficiently read ~200 graphs per instance

[aleSuglia]

Hi there,

In my use case, for each example of my dataset, I have ~200 graphs associated with it. Therefore, in my dataset reader __getitem__, I call dgl.load_graphs to load the stored information for each example. However, each call takes about ~0.3s (with peaks of 1.23s) which is quite a lot if I want to use a big batch size. Any suggestions about how to handle this use case?

[VoVAllen]

@aleSuglia

In my use case, for each example of my dataset, I have ~200 graphs associated with it. Therefore, in my dataset reader __getitem__, I call dgl.load_graphs to load the stored information for each example. However, each call takes about ~0.3s (with peaks of 1.23s) which is quite a lot if I want to use a big batch size. Any suggestions about how to handle this use case?

Hi,

How large is your graph?

[aleSuglia]

Each graph might have around 18 nodes. Each node has 3 set of features of different sizes (tensors).

[VoVAllen]

Thanks. It seems not ideal for DGL to take such long time to load those graphs. Could you file an issue at github, with a pseudo code of your scenario? Thanks