How to sample heterograph in HAN?

sakuraCY · September 23, 2020, 12:08pm

I have a question regading in Heterogeneous Graph Attention Network. The input graph is too large causing GPU out of memory.How can I split the graph into multiple subgraphs?

mufeili · September 24, 2020, 12:25am

You can use neighbor sampling for mini-batch training of HAN. There’s a section in DGL’s user guide for mini-batch training of GNN on heterogeneous graphs for node classification.

BarclayII · September 28, 2020, 8:13am

Further discussion in https://github.com/dmlc/dgl/issues/2248 .

stupidoge · August 28, 2023, 3:35am

@BarclayII Has anyone solved this problem? I have met similar issues when I try to use our function metapath_reachable_graph
this discussion still don’t have any solution:

github.com/dmlc/dgl

[Feature Request] Improve metapath_reachable_graph for large graphs

opened 03:56PM - 27 Sep 20 UTC

closed 06:20PM - 06 Oct 20 UTC

lingfanyu

## 🚀 Feature I suggest the following support for `dgl.metapath_reachable_grap…h`: - Let users specify source nodes, from which the output graph is reachable following given metapath - Add a neighbor sampling version of this API ## Motivation I am trying to run [Heterogeneous Graph Attention Network (HAN)](https://arxiv.org/pdf/1903.07293.pdf) on large graphs. The example [HAN implementation in DGL](https://github.com/dmlc/dgl/tree/master/examples/pytorch/han) works perfectly on small graphs like ACM (only 4k paper nodes). But I find it not easy to scale to larger datasets like [ogbn-mag](https://ogb.stanford.edu/docs/nodeprop/#ogbn-mag). ### The problem HAN defines neighborhood based on metapath. However, this step makes the graph a lot denser. Take ACM dataset as an example, there are about 4k papers, 70 fields, and 4k paper-field edges in the graph. The reachable graph of metapath paper-field-paper has about 2.2M edges. On large datasets, there are millions of nodes, and more edge types (resulting in more potential metapaths) and performing metapath_reachable_graph for full graph does not scale. ## Pitch I want two things to happen if this feature is added: - Mini-batch training for HAN instead of full-batch. This requires generating reachable graph from a batch of nodes provided by users - Performing neighbor sampling (maybe at each step in the metapath) to further reduce the size of generated reachable graph ## Alternatives One alternative way I can think of to implement what I need using DGL's current API is to use `dgl.sampling.random_walk`. For a mini-batch of nodes, it might be equivalent to performing `dgl.sampling.random_walk(g, nodes, metapath)` for K times. And then I can take only the last step of the returned paths to form a size K neighborhood for each node in the batch. Another way (less relevant to requested feature) to scale HAN to large graph is to design a subgraph sampling approach which needs to be aware of the heterogeneity of the graph and the user-specified metapaths. Then HAN can run on the sampled much smaller heterogeneous subgraph. @jermainewang @BarclayII @mufeili

minjie · August 30, 2023, 3:06pm

The original HAN algorithm is unfortunately not scalable on large graphs due to it’s tendency to significantly densify the graph. In overall, I don’t see a straightforward answer to that, which means it is probably an open question that requires research effort. As a result, I won’t treat it as a bugfix or a feature request. However, I would change my mind if there exist some work that present a reasonable upgrade (e.g., a mini-batch version) of HAN, and the team would be happy to learn from that.