GATConv layer slows down code due to following warning

I changed my SAGEConv layers to GAT layers (making sure to account for addition of heads etc.) and the code runs, but I get a warning at the start of the script
../torch/csrc/utils/python_arg_parser.cpp:756: UserWarning: This overload of nonzero is deprecated: nonzero(Tensor input, *, Tensor out) Consider using one of the following signatures instead: nonzero(Tensor input, *, bool as_tuple)

I googled this and it seems on regular PyTorch it is an issue that severely slows down code, but I’m not using nonzero anywhere in my code so I am guessing it is something in the GAT layer? Is there a fix for this?

We just released DGL 0.5. Can you try updating and see if the issue still exists? Note that you will need to first uninstall the old DGL versions.

I don’t get the warning anymore, thanks!

However, I got the new warning
DGLWarning: Recommend creating graphs by dgl.graph(data) instead of dgl.DGLGraph(data)
what is the main difference between the old class and the new class?

Hi, there is no difference in terms of both correctness and performance, but we are moving away from creating a DGLGraph directly using its constructor for project maintenance purposes.

1 Like

Thanks!

Apologies but I just got another warning. This time it was

DGLWarning: DGLGraph.__len__ is deprecated.Please directly call DGLGraph.number_of_nodes. return warnings.warn(message, category=category, stacklevel=1)

however, I don’t take the Len of a DGL graph anywhere (and in fact I don’t use dgl.DGLGraph, I change it to dgl.graph) so is this more backend stuff?

I have got the same warning even with DGL 0.5.2. It’s very irritating.

/home/usr/.local/lib/python3.6/site-packages/dgl/base.py:45: DGLWarning: DGLGraph.len is deprecated.Please directly call DGLGraph.number_of_nodes.
return warnings.warn(message, category=category, stacklevel=1)

How did you trigger the warning? Is this from your code base or DGL’s code base? If it’s the former one, you can avoid the warning by replacing len with num_nodes

running this code will give there error still, I believe it is internet to DGL’s code base.

import dgl
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import networkx as nx
from dgl.nn.pytorch import GraphConv
import itertools


def build_karate_club_graph():
    # All 78 edges are stored in two numpy arrays. One for source endpoints
    # while the other for destination endpoints.
    src = np.array([1, 2, 2, 3, 3, 3, 4, 5, 6, 6, 6, 7, 7, 7, 7, 8, 8, 9, 10, 10,
        10, 11, 12, 12, 13, 13, 13, 13, 16, 16, 17, 17, 19, 19, 21, 21,
        25, 25, 27, 27, 27, 28, 29, 29, 30, 30, 31, 31, 31, 31, 32, 32,
        32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33,
        33, 33, 33, 33, 33, 33, 33, 33, 33, 33])
    dst = np.array([0, 0, 1, 0, 1, 2, 0, 0, 0, 4, 5, 0, 1, 2, 3, 0, 2, 2, 0, 4,
        5, 0, 0, 3, 0, 1, 2, 3, 5, 6, 0, 1, 0, 1, 0, 1, 23, 24, 2, 23,
        24, 2, 23, 26, 1, 8, 0, 24, 25, 28, 2, 8, 14, 15, 18, 20, 22, 23,
        29, 30, 31, 8, 9, 13, 14, 15, 18, 19, 20, 22, 23, 26, 27, 28, 29, 30,
        31, 32])
    # Edges are directional in DGL; Make them bi-directional.
    u = np.concatenate([src, dst])
    v = np.concatenate([dst, src])
    # Construct a DGLGraph
    return dgl.graph((u, v))


G = build_karate_club_graph()
print('We have %d nodes.' % G.number_of_nodes())
print('We have %d edges.' % G.number_of_edges())

# Since the actual graph is undirected, we convert it for visualization
# purpose.
nx_G = G.to_networkx().to_undirected()
# Kamada-Kawaii layout usually looks pretty for arbitrary graphs
pos = nx.kamada_kawai_layout(nx_G)
nx.draw(nx_G, pos, with_labels=True, node_color=[[.7, .7, .7]])

# In DGL, you can add features for all nodes at once, using a feature tensor that
# batches node features along the first dimension. The code below adds the learnable
# embeddings for all nodes:


embed = nn.Embedding(34, 5)  # 34 nodes with embedding dim equal to 5
G.ndata['feat'] = embed.weight

# print out node 2's input feature
print(G.ndata['feat'][2])

# print out node 10 and 11's input features
print(G.ndata['feat'][[10, 11]])


class GCN(nn.Module):
    def __init__(self, in_feats, hidden_size, num_classes):
        super(GCN, self).__init__()
        self.conv1 = GraphConv(in_feats, hidden_size)
        self.conv2 = GraphConv(hidden_size, num_classes)

    def forward(self, g, inputs):
        h = self.conv1(g, inputs)
        h = torch.relu(h)
        h = self.conv2(g, h)
        return h


# The first layer transforms input features of size of 5 to a hidden size of 5.
# The second layer transforms the hidden layer and produces output features of
# size 2, corresponding to the two groups of the karate club.
net = GCN(5, 5, 2)

inputs = embed.weight
labeled_nodes = torch.tensor([0, 33])  # only the instructor and the president nodes are labeled
labels = torch.tensor([0, 1])  # their labels are different

optimizer = torch.optim.Adam(itertools.chain(net.parameters(), embed.parameters()), lr=0.01)
all_logits = []
for epoch in range(50):
    logits = net(G, inputs)
    # we save the logits for visualization later
    all_logits.append(logits.detach())
    logp = F.log_softmax(logits, 1)
    # we only compute loss for labeled nodes
    loss = F.nll_loss(logp[labeled_nodes], labels)

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    print('Epoch %d | Loss: %.4f' % (epoch, loss.item()))

I am not able to reproduce the warning with your code snippet. What’s the DGL version?

DGL version 0.5.2.

The exact error warning is /Users/username/opt/anaconda3/envs/misc/lib/python3.7/site-packages/dgl/base.py:45: DGLWarning: DGLGraph.__len__ is deprecated.Please directly call DGLGraph.number_of_nodes. return warnings.warn(message, category=category, stacklevel=1)

I tried DGL 0.5.2, but still failed to reproduce the error. Can you try reinstalling DGL? Which OS do you use? How did you install DGL?

Yes, I have tried uninstalling and reinstalling. I just did again, and still get the error. I work on Mac OS but also have a virtual machine workstation that runs on linux where the error also occurs. I install using pip install dgl from the command line in the Conda environment I use. It is also DGL 0.5.2 that is/was installed as when I pip uninstall dgl I get asked to confirm whether I want to uninstall DGL 0.5.2.

I’m still unable to reproduce the warning. What’s your PyTorch version?

the PyTorch version I am using is 1.7.0

I have the same warning displayed for any DGL command and any python command. Don’t know what is wrong. I had uninstalled my previous version as well.

/home/user/.local/lib/python3.6/site-packages/dgl/base.py:45: DGLWarning: DGLGraph.len is deprecated.Please directly call DGLGraph.number_of_nodes.
return warnings.warn(message, category=category, stacklevel=1)

I have to use the warnings library to supress it as
import warnings
warnings.filterwarnings("ignore")

How I installed DGL - pip install dgl-cu101
Ubuntu 16.04
Python3.6
DGL 0.5.2

NOTE: I found out that the warning only appears when I run the command in Spyder Console, any command, throws warning

import sys
/home/user/.local/lib/python3.6/site-packages/dgl/base.py:45: DGLWarning: DGLGraph.len is deprecated.Please directly call DGLGraph.number_of_nodes.
return warnings.warn(message, category=category, stacklevel=1)

Hello, @mufeili , I have the same warning.

E:\Python36\lib\site-packages\dgl\base.py:45: DGLWarning: DGLGraph.len is deprecated.Please directly call DGLGraph.number_of_nodes.
return warnings.warn(message, category=category, stacklevel=1)

My previous version of DGL is 0.4, and there is no warning. Since I upgrade DGL to 0.6, I have the above warning.
NOTE1: In pycharm, DEBUG code have the above warning, but RUN code do not have the above warning.
NOTE2: Is there a way to install the previous version of dgl using pip?

DGL 0.6’s GATConv’s performance is much much better than DGL 0.4’s and I don’t recommend you to use DGL 0.4

Thank you for your advice.
I am looking at the difference between v0.4 and v0.6, and find that the example in v0.4 uses dgl.DGLGraph() to create a graph (DGL at a Glance — DGL 0.4.3post2 documentation), while v0.6 uses dgl.graph() to create a graph (DGL at a Glance — DGL 0.6.0 documentation). What is the difference between them?

We simplified the graph creation API as dgl.graph(...), and the returned object has type DGLGraph.
btw, since DGL 0.5 on our DGLGraph class support more than one edge type and DGL 0.4’s DGLGraph support only one edge type.

@mufeili I am still receiving this error on DGL 0.6. I confirmed I am running DGL 0.6 by running dgl.__version__ and I had installed it by first pip uninstalling dgl and then pip installing dgl. I get the error just when making a dgl graph object from a networkx graph by using g = dgl.from_networkx(graph). The exact error I get is DGLWarning: DGLGraph.__len__ is deprecated.Please directly call DGLGraph.number_of_nodes. return warnings.warn(message, category=category, stacklevel=1)