GATConv layer slows down code due to following warning

I changed my SAGEConv layers to GAT layers (making sure to account for addition of heads etc.) and the code runs, but I get a warning at the start of the script
../torch/csrc/utils/python_arg_parser.cpp:756: UserWarning: This overload of nonzero is deprecated: nonzero(Tensor input, *, Tensor out) Consider using one of the following signatures instead: nonzero(Tensor input, *, bool as_tuple)

I googled this and it seems on regular PyTorch it is an issue that severely slows down code, but I’m not using nonzero anywhere in my code so I am guessing it is something in the GAT layer? Is there a fix for this?

We just released DGL 0.5. Can you try updating and see if the issue still exists? Note that you will need to first uninstall the old DGL versions.

I don’t get the warning anymore, thanks!

However, I got the new warning
DGLWarning: Recommend creating graphs by dgl.graph(data) instead of dgl.DGLGraph(data)
what is the main difference between the old class and the new class?

Hi, there is no difference in terms of both correctness and performance, but we are moving away from creating a DGLGraph directly using its constructor for project maintenance purposes.

1 Like

Thanks!

Apologies but I just got another warning. This time it was

DGLWarning: DGLGraph.__len__ is deprecated.Please directly call DGLGraph.number_of_nodes. return warnings.warn(message, category=category, stacklevel=1)

however, I don’t take the Len of a DGL graph anywhere (and in fact I don’t use dgl.DGLGraph, I change it to dgl.graph) so is this more backend stuff?

I have got the same warning even with DGL 0.5.2. It’s very irritating.

/home/usr/.local/lib/python3.6/site-packages/dgl/base.py:45: DGLWarning: DGLGraph.len is deprecated.Please directly call DGLGraph.number_of_nodes.
return warnings.warn(message, category=category, stacklevel=1)

How did you trigger the warning? Is this from your code base or DGL’s code base? If it’s the former one, you can avoid the warning by replacing len with num_nodes

running this code will give there error still, I believe it is internet to DGL’s code base.

import dgl
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import networkx as nx
from dgl.nn.pytorch import GraphConv
import itertools


def build_karate_club_graph():
    # All 78 edges are stored in two numpy arrays. One for source endpoints
    # while the other for destination endpoints.
    src = np.array([1, 2, 2, 3, 3, 3, 4, 5, 6, 6, 6, 7, 7, 7, 7, 8, 8, 9, 10, 10,
        10, 11, 12, 12, 13, 13, 13, 13, 16, 16, 17, 17, 19, 19, 21, 21,
        25, 25, 27, 27, 27, 28, 29, 29, 30, 30, 31, 31, 31, 31, 32, 32,
        32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33,
        33, 33, 33, 33, 33, 33, 33, 33, 33, 33])
    dst = np.array([0, 0, 1, 0, 1, 2, 0, 0, 0, 4, 5, 0, 1, 2, 3, 0, 2, 2, 0, 4,
        5, 0, 0, 3, 0, 1, 2, 3, 5, 6, 0, 1, 0, 1, 0, 1, 23, 24, 2, 23,
        24, 2, 23, 26, 1, 8, 0, 24, 25, 28, 2, 8, 14, 15, 18, 20, 22, 23,
        29, 30, 31, 8, 9, 13, 14, 15, 18, 19, 20, 22, 23, 26, 27, 28, 29, 30,
        31, 32])
    # Edges are directional in DGL; Make them bi-directional.
    u = np.concatenate([src, dst])
    v = np.concatenate([dst, src])
    # Construct a DGLGraph
    return dgl.graph((u, v))


G = build_karate_club_graph()
print('We have %d nodes.' % G.number_of_nodes())
print('We have %d edges.' % G.number_of_edges())

# Since the actual graph is undirected, we convert it for visualization
# purpose.
nx_G = G.to_networkx().to_undirected()
# Kamada-Kawaii layout usually looks pretty for arbitrary graphs
pos = nx.kamada_kawai_layout(nx_G)
nx.draw(nx_G, pos, with_labels=True, node_color=[[.7, .7, .7]])

# In DGL, you can add features for all nodes at once, using a feature tensor that
# batches node features along the first dimension. The code below adds the learnable
# embeddings for all nodes:


embed = nn.Embedding(34, 5)  # 34 nodes with embedding dim equal to 5
G.ndata['feat'] = embed.weight

# print out node 2's input feature
print(G.ndata['feat'][2])

# print out node 10 and 11's input features
print(G.ndata['feat'][[10, 11]])


class GCN(nn.Module):
    def __init__(self, in_feats, hidden_size, num_classes):
        super(GCN, self).__init__()
        self.conv1 = GraphConv(in_feats, hidden_size)
        self.conv2 = GraphConv(hidden_size, num_classes)

    def forward(self, g, inputs):
        h = self.conv1(g, inputs)
        h = torch.relu(h)
        h = self.conv2(g, h)
        return h


# The first layer transforms input features of size of 5 to a hidden size of 5.
# The second layer transforms the hidden layer and produces output features of
# size 2, corresponding to the two groups of the karate club.
net = GCN(5, 5, 2)

inputs = embed.weight
labeled_nodes = torch.tensor([0, 33])  # only the instructor and the president nodes are labeled
labels = torch.tensor([0, 1])  # their labels are different

optimizer = torch.optim.Adam(itertools.chain(net.parameters(), embed.parameters()), lr=0.01)
all_logits = []
for epoch in range(50):
    logits = net(G, inputs)
    # we save the logits for visualization later
    all_logits.append(logits.detach())
    logp = F.log_softmax(logits, 1)
    # we only compute loss for labeled nodes
    loss = F.nll_loss(logp[labeled_nodes], labels)

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    print('Epoch %d | Loss: %.4f' % (epoch, loss.item()))

I am not able to reproduce the warning with your code snippet. What’s the DGL version?

DGL version 0.5.2.

The exact error warning is /Users/username/opt/anaconda3/envs/misc/lib/python3.7/site-packages/dgl/base.py:45: DGLWarning: DGLGraph.__len__ is deprecated.Please directly call DGLGraph.number_of_nodes. return warnings.warn(message, category=category, stacklevel=1)

I tried DGL 0.5.2, but still failed to reproduce the error. Can you try reinstalling DGL? Which OS do you use? How did you install DGL?

Yes, I have tried uninstalling and reinstalling. I just did again, and still get the error. I work on Mac OS but also have a virtual machine workstation that runs on linux where the error also occurs. I install using pip install dgl from the command line in the Conda environment I use. It is also DGL 0.5.2 that is/was installed as when I pip uninstall dgl I get asked to confirm whether I want to uninstall DGL 0.5.2.

I’m still unable to reproduce the warning. What’s your PyTorch version?

the PyTorch version I am using is 1.7.0

I have the same warning displayed for any DGL command and any python command. Don’t know what is wrong. I had uninstalled my previous version as well.

/home/user/.local/lib/python3.6/site-packages/dgl/base.py:45: DGLWarning: DGLGraph.len is deprecated.Please directly call DGLGraph.number_of_nodes.
return warnings.warn(message, category=category, stacklevel=1)

I have to use the warnings library to supress it as
import warnings
warnings.filterwarnings("ignore")

How I installed DGL - pip install dgl-cu101
Ubuntu 16.04
Python3.6
DGL 0.5.2

NOTE: I found out that the warning only appears when I run the command in Spyder Console, any command, throws warning

import sys
/home/user/.local/lib/python3.6/site-packages/dgl/base.py:45: DGLWarning: DGLGraph.len is deprecated.Please directly call DGLGraph.number_of_nodes.
return warnings.warn(message, category=category, stacklevel=1)