ModuleNotFoundError in DGL with PyTorch 2.2 on Python 3.12 Docker Image

Questions
#5

How did you install DGL when you got this error?

If you have the CPU version of torch 2.3, then you need to use this link to install DGL: https://data.dgl.ai/wheels/torch-2.3/repo.html

#6

What about GPU version? Thanks.

#7

Both of the lines below are only for torch 2.3 versions.

pip install dgl -f https://data.dgl.ai/wheels/torch-2.3/cu121/repo.html

For CUDA 12.1 and

pip install dgl -f https://data.dgl.ai/wheels/torch-2.3/cu118/repo.html

for CUDA 11.8.

#8

Thank you, then I got exactly the same error as the previous user posted.

#9

I can further report that, after install, import dgl gets the following error:
ImportError: Cannot load Graphbolt C++ library
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-242fa303-4a84-4d3b-aec6-3a2e19f10b8c/lib/python3.10/site-packages/dgl/graphbolt/init.py:31, in load_graphbolt()
30 try:
—> 31 torch.classes.load_library(path)
32 except Exception: # pylint: disable=W0703

I have checked the torch and cuda version should match the installation command you recommended:
%pip install torch==2.3.0
%pip install dgl -f https://data.dgl.ai/wheels/torch-2.3/cu121/repo.html

Any advise? Thank you.

#10

are sure you’re isntalling cuda version of torch? could you try to install with pip install torch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 --index-url https://download.pytorch.org/whl/cu121 and check with torch.cuda.is_available()?

#11

After running:
%pip install torch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 --index-url https://download.pytorch.org/whl/cu121
%pip install dgl -f https://data.dgl.ai/wheels/torch-2.3/cu121/repo.html
torch.cuda.is_available() still returns false somehow…

#12

I just ran pip install -r requirements.txt, using the same requirements I posted originally but replacing 2.3 for pytorch and DGL

#13

DGL 2.4 is out and it has support for torch 2.4.1. Would you like to try out this combination?

#14

sure, if not too much trouble, what is the command to install dgl 2.4 if we use torch 2.4.1 and cuda 12.1?

#15

CUDA: pip install dgl -f https://data.dgl.ai/wheels/torch-2.4/cu124/repo.html

CPU: pip install dgl -f https://data.dgl.ai/wheels/torch-2.4/repo.html

#16

Thank you, could we use cu121 instead?

#17

Yes. If you go to the DGL webpage to the getting started section, you can pick cuda version, torch version and conda/pip etc.

https://www.dgl.ai/pages/start.html

#18

Thank you. I installed dgl 2.4 and torch 2.4 But

import dgl

still gives error:
FileNotFoundError: Cannot find DGL C++ graphbolt library at /local_disk0/.ephemeral_nfs/envs/pythonEnv-525aecaa-dec3-4609-a10d-dbd14eef7889/lib/python3.10/site-packages/dgl/graphbolt/libgraphbolt_pytorch_2.4.0.so

#19

I downloaded the wheel and the file is there:

image
image1187×581 43.2 KB

#20

I recommend going to the installation directory and verifying if the .so files can be found there.

#21

unfortunately I’m using databrick where I cannot access the installed directory, hence it might not be possible to check if the file is there or not.

#22

For future reference, if someone is using databrick system, dgl works when downgrade to torch 2.1.0+cuda 11.8.

1 Like
#23

I managed to resolve my issue. Basically (and I don’t know why) if we just use the requirements.txt to tell pip the packages to install it installs the wrong versions. Here is my final Dockerfile:

FROM public.ecr.aws/lambda/python:3.12
ARG PRIVATE_PIP_INDEX_URL
ARG PROD_REGION
ARG PROD_ACCESS_KEY
ARG PROD_SECRET_ACCESS_KEY
ENV DGLBACKEND=pytorch

# Install the dependencies
COPY requirements.txt .
RUN pip install torch==2.3.1 \
    --index-url https://download.pytorch.org/whl/cpu

RUN pip install dgl -f https://data.dgl.ai/wheels/torch-2.3/repo.html
RUN pip3 install \
    --index-url  https://pypi.org/simple \
    --extra-index-url "${PRIVATE_PIP_INDEX_URL}" \
    -r requirements.txt --target "${LAMBDA_TASK_ROOT}"

# Copy our code
COPY *.py ${LAMBDA_TASK_ROOT}


# Set the handler function
CMD [ "evaluate_model.handler" ]%
1 Like
#24

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.