I am generating thousands of graphs and using dgl.save_graphs to save them, but I’d like to be able to add them one at a time, rather than having to keep all the graphs in my program memory simultaneously and then dump them to the file, as I get OOM errors. I would also like to be able to read one graph at a time from the file, without having to load all of them into memory like dgl.load_graphs. How can I do this?
Is there no efficient way to do this?
I don’t think it can be supported by the current functionality. For now, I think the only solution is to store them in separate files. How big is your graph?
100k graphs with 20-300 nodes each.
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.