GAT layers in HeteroGraphs ('GATConv' object has no attribute 'fc' )

[pcrocker]

I am trying to use a series of GATlayers to connect a series of unidirectional bipartite graphs. My graph looks like a binary tree where each layer is an edge type and the edges flow up to the root.

4nodesTypeA → 2nodesTypeB → 1nodeTypeC

data_dict = {}
data_dict[('A', 'L0', 'B')] = ([],[])
data_dict[('B', 'L1', 'C')] = ([],[])
num_nodes_dict = {'A':4, 'B': 2, 'C':1}
base_graph = dgl.heterograph(data_dict, num_nodes_dict)

base_graph.add_edges(torch.tensor([0,1,2,3]), torch.tensor([0,0,1,1]), etype='L0')
base_graph.add_edges(torch.tensor([0,1]), torch.tensor([0,0]), etype='L1')

Now, I am trying to do a HeteroGraphConv on this graph via:

conv = dgl.nn.HeteroGraphConv({
    'L0': dglnn.GATConv((2,4),4,1),
    'L1': dglnn.GATConv((4,8),8,1),
})

I believe this makes the dim of typeA: 2, typeB:4, typeC:8. If I then initialize ONLY TYPE A and run the HeteroConv, I expect it to populate the typeB. If I run it again on this output, type C should be populated.

a_feats = torch.rand((4,2))
out = conv(base_graph, a_feats)
out_2 = conv(base_graph, out)

Here is where I get my first error
This returns: ‘GATConv’ object has no attribute ‘fc’

Any thoughts here? This seems like an import error. If I am able to fix it, does this method make sense? Does it seem possible?

[mufeili]

Why did you add edges after creating an empty graph rather than creating the graph with data_dict at once?

When you instantiate a GATConv object, if you pass a 2-tuple to the in_feats argument, then it will initialize self.fc_src and self.fc_dst separately for the source and destination nodes rather than initialize self.fc for both the source and destination nodes. See here. When you use HeteroGraphConv for some edge types, it will invoke message passing on the edge types in parallel and you don’t need to call it sequentially. If you want to populate the typeB using typeA, you should create independent GATConv instances and manually do that.

[pcrocker]

Thanks Mufeili. I add the edges after for readability reasons. In my code I add a lot more edges in batches. Is there any reason not to do this?

I see what you are saying. That source code is helpful. In my case, I have 3 Node types and 2 edge types all in the same graph. It is essentially two bipartite graphs with edges from typeA->typeB and typeB->typeC. Where really I have no messages being passed from typeB->typeA, I must initialize the GAT like I do. Is it true then, that when calling a GAT with a 2-tuple as the in_feats, the inputs can start with different feature sizes but both end with feature size out_feats?

Is there a way to call a GATConv instance on only one edge type in a graph that is not by using a HeteroGraphConv? I want to do a pooling operation similar to DiffPooling. The bottom layer, typeA, will feed and populate typeB on the first pass. On the second pass, typeB will feed and populate typeC. Would it be better if I broke my graph into two distinct bipartite graphs? Or is it okay to just pass it through the HeteroGraphConv twice, where the first pass input is (TypeA, TypeB(zeros)), and the second pass input is (TypeB(from first pass), TypeC(zeros)).

[mufeili]

In my code I add a lot more edges in batches. Is there any reason not to do this?

Frequently modifying graph structures can be costly unless you need to dynamically update graphs.

Is it true then, that when calling a GAT with a 2-tuple as the in_feats , the inputs can start with different feature sizes but both end with feature size out_feats?

The input feature size can be different for the source and destination nodes and GATConv will handle that by nature. You just need to specify the feature size with a 2-tuple in instantiating the conv object and pass a tuple of node features in forward.

Is there a way to call a GATConv instance on only one edge type in a graph that is not by using a HeteroGraphConv?

You can slice the graph for a particular relation with g[etype] like here. I don’t think you need HeteroGraphConv. Just use two separate GATConv modules.

[minjie]

On a side note, we recently fixed a bug in HeteroGraphConv. Would you mind install the latest prelease and see if the problem resolved?

[BarclayII]

@pcrocker

a_feats = torch.rand((4,2))
out = conv(base_graph, a_feats)
out_2 = conv(base_graph, out)

To add to @mufeili’s comment, HeteroGraphConv expects a dictionary of node types and features, so you need something like:

out = conv(base_graph, {'A': a_feats, 'B': b_feats, 'C': c_feats})

If you only expect to apply GAT for one edge type at a time, then you could slice the graph by relation as @mufeili explained.

[pcrocker]

Thank you all for the help. In reality, I had multiple edge types at each layer (L0 was a supertype and L0a, L0b… were subtypes), so I ended up splitting the graph by supertype using base_graph.edge_type_subgraph([L0a, L0b…]) and calling a HeteroGraphConv on that split with a GAT for each subtype. I then cascaded these split/call passes as suggested by @mufeili except with these HeteroGraphConv layers instead of GATConv layers. So far so good! Thank you all for the quick support.