I’m not able to load any graph on GPU.
- GPU: A100
- Cuda version: 11.3 (confirmed with
nvcc --version
) - Pytorch version: 1.11.0
- Python version: tried 3.8 and 3.9
- Package manager: miniconda
- Command used to download dgl:
conda install -c dglteam dgl-cuda11.3
It works fine with Pytorch.
Failing code and error:
torch.cuda.is_available()
# True
import dgl
import torch
from ogb.nodeproppred import DglNodePropPredDataset
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
dataset = DglNodePropPredDataset('ogbn-arxiv')
dataset[0][0].to(device)
DGLError: [21:03:58] /opt/dgl/src/runtime/cuda/cuda_device_api.cc:97: Check failed: e == cudaSuccess || e == cudaErrorCudartUnloading: CUDA: the provided PTX was compiled with an unsupported toolchain.
Stack trace:
[bt] (0) /home/miniconda3/envs/gnn-p38/lib/python3.8/site-packages/dgl/libdgl.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x4f) [0x7f3b93fde57f]
[bt] (1) /home/miniconda3/envs/gnn-p38/lib/python3.8/site-packages/dgl/libdgl.so(dgl::runtime::CUDADeviceAPI::AllocDataSpace(DLContext, unsigned long, unsigned long, DLDataType)+0x108) [0x7f3b944b6828]
[bt] (2) /home/miniconda3/envs/gnn-p38/lib/python3.8/site-packages/dgl/libdgl.so(dgl::runtime::NDArray::Empty(std::vector<long, std::allocator<long> >, DLDataType, DLContext)+0x351) [0x7f3b94324b51]
[bt] (3) /home/miniconda3/envs/gnn-p38/lib/python3.8/site-packages/dgl/libdgl.so(dgl::runtime::NDArray::CopyTo(DLContext const&, void* const&) const+0xc7) [0x7f3b9435ed87]
[bt] (4) /home/miniconda3/envs/gnn-p38/lib/python3.8/site-packages/dgl/libdgl.so(dgl::UnitGraph::CopyTo(std::shared_ptr<dgl::BaseHeteroGraph>, DLContext const&, void* const&)+0x2ff) [0x7f3b9447985f]
[bt] (5) /home/miniconda3/envs/gnn-p38/lib/python3.8/site-packages/dgl/libdgl.so(dgl::HeteroGraph::CopyTo(std::shared_ptr<dgl::BaseHeteroGraph>, DLContext const&, void* const&)+0x109) [0x7f3b94370979]
[bt] (6) /home/miniconda3/envs/gnn-p38/lib/python3.8/site-packages/dgl/libdgl.so(+0x6aec89) [0x7f3b9437dc89]
[bt] (7) /home/miniconda3/envs/gnn-p38/lib/python3.8/site-packages/dgl/libdgl.so(DGLFuncCall+0x48) [0x7f3b94302ea8]
[bt] (8) /home/miniconda3/envs/gnn-p38/lib/python3.8/site-packages/dgl/_ffi/_cy3/core.cpython-38-x86_64-linux-gnu.so(+0x16fb9) [0x7f3b93c32fb9]