Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hi dataiku community!
I am trying to use Dataiku's Object Detection example project to train my own model: Project description - Wiki | Dataiku
However, I am having issues when attempting to create the suggested code env: py_38_sample_object_detection (Python) - Code Envs | Dataiku for container configurations
I have created an code env exactly equivalent to the above linked, but when I choose:
Containerized execution --> Build for selected container configurations
I keep getting the same error:
docker failed (exit code: 1)""
torch==1.8.1torchvision==0.9.1tornado==5.1.1tqdm==4.65.0traitlets==4.3.3typed-ast==1.5.4typing_extensions==4.6.3urllib3==1.26.6wcwidth==0.2.6Werkzeug==2.2.3yacs==0.1.8zipp==3.15.0Installed kernelspec py-dku-venv-object-detection-new in /data/dataiku/dss_data/jupyter-run/jupyter/kernels/py-dku-venv-object-detection-newInstalled kernelspec py-dku-containerized-venv-object-detection-new-compute-design-gpu in /data/dataiku/dss_data/jupyter-run/jupyter/kernels/py-dku-containerized-venv-object-detection-new-compute-design-gpuAdding resources environment variables to Jupyter kernel.json ...Building docker image$ docker build -t dataiku-container-exec-base-cuda-cicd-11.2.0-pyenv-object-detection-new:r-2023-06-23-13-28-28-264 --force-rm /data/dataiku/dss_data/tmp/docker-image/codeenv-object-detection-new6461684965385574816 -t crk8sdataikuprod.azurecr.io/dataiku-container-exec-base-cuda-cicd-11.2.0-pyenv-object-detection-new:r-2023-06-23-13-28-28-264Sending build context to Docker daemon 4.096kBStep 1/13 : FROM dataiku-container-exec-base-cuda-cicd:11.2.0 ---> 382ce570ed33Step 2/13 : USER root ---> Using cache ---> b9b2e9f51916Step 3/13 : WORKDIR /opt/dataiku ---> Using cache ---> 5627184cd74cStep 4/13 : ENV PYTHONPATH= ---> Using cache ---> 5a1effc1f7acStep 5/13 : ENV R_LIBS_USER= ---> Using cache ---> 51131f18b387Step 6/13 : ENV DKU_IMAGE_BUILD_TIMESTAMP=1687526908264 ---> Running in 3f6026947273Removing intermediate container 3f6026947273 ---> 07f6b26d6c01Step 7/13 : RUN ["bin/python", "build/virtualenv.pyz", "-p", "python3.7", "code-env"] ---> Running in 168348aa0cc5created virtual environment CPython3.7.15.final.0-64 in 650ms creator CPython3Posix(dest=/opt/dataiku/code-env, clear=False, no_vcs_ignore=False, global=False) seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv) added seed packages: pip==22.1.2, setuptools==62.6.0, wheel==0.37.1 activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivatorRemoving intermediate container 168348aa0cc5 ---> 50dd2916aff0Step 8/13 : COPY code-env/pip.packages.txt code-env/ ---> c5b877cf51e1Step 9/13 : RUN ["/opt/dataiku/code-env/bin/python", "-m", "pip", "install", "--extra-index-url", "https://amrab%40orsted.com:cmVmdGtuOjAxOjE3MTYwMjAyMDg6ZGRNMEVpVkdyNUM5QzRZY3NuQ2JKUkcxTlNj@orsted.jfrog.io/artifactory/incubator-pytorch-remote", "-r", "code-env/pip.packages.txt"] ---> Running in 4eaf662bfd5fLooking in indexes: https://pypi.org/simple, https://amrab%40orsted.com:****@orsted.jfrog.io/artifactory/incubator-pytorch-remoteCollecting git+https://github.com/facebookresearch/detectron2.git (from -r code-env/pip.packages.txt (line 8)) Cloning https://github.com/facebookresearch/detectron2.git to /tmp/pip-req-build-cmdt0cyg[91m Running command git clone --quiet https://github.com/facebookresearch/detectron2.git /tmp/pip-req-build-cmdt0cyg[0m Resolved https://github.com/facebookresearch/detectron2.git to commit eddac5d5b4fce9b5eba5518f9ee348fa3a531407 Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'error'[91m error: subprocess-exited-with-error ร python setup.py egg_info did not run successfully. โ exit code: 1 โฐโ> [6 lines of output] Traceback (most recent call last): File "<string>", line 36, in <module> File "<pip-setuptools-caller>", line 34, in <module> File "/tmp/pip-req-build-cmdt0cyg/setup.py", line 10, in <module> import torch ModuleNotFoundError: No module named 'torch' [end of output] note: This error originates from a subprocess, and is likely not a problem with pip.[0m[91merror: metadata-generation-failedร Encountered error while generating package metadata.โฐโ> See above for output.note: This is an issue with the package mentioned above, not pip.hint: See above for details.[0m[notice] A new release of pip available: 22.1.2 -> 23.1.2[notice] To update, run: /opt/dataiku/code-env/bin/python -m pip install --upgrade pipRemoving intermediate container 4eaf662bfd5fThe command '/opt/dataiku/code-env/bin/python -m pip install --extra-index-url https://amrab%40orsted.com:cmVmdGtuOjAxOjE3MTYwMjAyMDg6ZGRNMEVpVkdyNUM5QzRZY3NuQ2JKUkcxTlNj@orsted.jfrog.io/artifactory/incubator-pytorch-remote -r code-env/pip.packages.txt' returned a non-zero code: 1
Does anybody know how to solve this? it works without the container figurations...
Operating system used: Windows 10
Hi Simone,
From the lines, you shared :
line 10, in <module> import torch ModuleNotFoundError: No module named 'torch'
It's failing when trying to install git+https://github.com/facebookresearch/detectron2.git
Changing this to https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.8/detectron2-0.5%2Bcu101-cp38-cp38-lin... instead seems to work fine
torch==1.8.1
torchvision==0.9.1
pyyaml==5.1
opencv-python==4.6.0.66
dash==2.6.1
dash-bootstrap-components==1.2.1
https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.8/detectron2-0.5%2Bcu101-cp38-cp38-linux_x86_64.whl
It appears this was broken upstream based on this threads so when this originally created it would have worked but no longer work : https://github.com/facebookresearch/detectron2/issues/4386