Object detection code env: Could not build container image for environment

simeh

Hi dataiku community!

I am trying to use Dataiku's Object Detection example project to train my own model: Project description - Wiki | Dataiku

However, I am having issues when attempting to create the suggested code env: py_38_sample_object_detection (Python) - Code Envs | Dataiku for container configurations

I have created an code env exactly equivalent to the above linked, but when I choose:

Containerized execution --> Build for selected container configurations

I keep getting the same error:

""Could not build container image for environment

docker failed (exit code: 1)""

torch==1.8.1torchvision==0.9.1tornado==5.1.1tqdm==4.65.0traitlets==4.3.3typed-ast==1.5.4typing_extensions==4.6.3urllib3==1.26.6wcwidth==0.2.6Werkzeug==2.2.3yacs==0.1.8zipp==3.15.0Installed kernelspec py-dku-venv-object-detection-new in /data/dataiku/dss_data/jupyter-run/jupyter/kernels/py-dku-venv-object-detection-newInstalled kernelspec py-dku-containerized-venv-object-detection-new-compute-design-gpu in /data/dataiku/dss_data/jupyter-run/jupyter/kernels/py-dku-containerized-venv-object-detection-new-compute-design-gpuAdding resources environment variables to Jupyter kernel.json ...Building docker image$ docker build -t dataiku-container-exec-base-cuda-cicd-11.2.0-pyenv-object-detection-new:r-2023-06-23-13-28-28-264 --force-rm /data/dataiku/dss_data/tmp/docker-image/codeenv-object-detection-new6461684965385574816 -t crk8sdataikuprod.azurecr.io/dataiku-container-exec-base-cuda-cicd-11.2.0-pyenv-object-detection-new:r-2023-06-23-13-28-28-264Sending build context to Docker daemon  4.096kBStep 1/13 : FROM dataiku-container-exec-base-cuda-cicd:11.2.0 ---> 382ce570ed33Step 2/13 : USER root ---> Using cache ---> b9b2e9f51916Step 3/13 : WORKDIR /opt/dataiku ---> Using cache ---> 5627184cd74cStep 4/13 : ENV PYTHONPATH= ---> Using cache ---> 5a1effc1f7acStep 5/13 : ENV R_LIBS_USER= ---> Using cache ---> 51131f18b387Step 6/13 : ENV DKU_IMAGE_BUILD_TIMESTAMP=1687526908264 ---> Running in 3f6026947273Removing intermediate container 3f6026947273 ---> 07f6b26d6c01Step 7/13 : RUN ["bin/python", "build/virtualenv.pyz", "-p", "python3.7", "code-env"] ---> Running in 168348aa0cc5created virtual environment CPython3.7.15.final.0-64 in 650ms  creator CPython3Posix(dest=/opt/dataiku/code-env, clear=False, no_vcs_ignore=False, global=False)  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)    added seed packages: pip==22.1.2, setuptools==62.6.0, wheel==0.37.1  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivatorRemoving intermediate container 168348aa0cc5 ---> 50dd2916aff0Step 8/13 : COPY code-env/pip.packages.txt code-env/ ---> c5b877cf51e1Step 9/13 : RUN ["/opt/dataiku/code-env/bin/python", "-m", "pip", "install", "--extra-index-url", "https://amrab@orsted.com:cmVmdGtuOjAxOjE3MTYwMjAyMDg6ZGRNMEVpVkdyNUM5QzRZY3NuQ2JKUkcxTlNj@orsted.jfrog.io/artifactory/incubator-pytorch-remote", "-r", "code-env/pip.packages.txt"] ---> Running in 4eaf662bfd5fLooking in indexes: https://pypi.org/simple, https://amrab@orsted.com:****@orsted.jfrog.io/artifactory/incubator-pytorch-remoteCollecting git+https://github.com/facebookresearch/detectron2.git (from -r code-env/pip.packages.txt (line 8))  Cloning https://github.com/facebookresearch/detectron2.git to /tmp/pip-req-build-cmdt0cyg&#27;[91m  Running command git clone --quiet https://github.com/facebookresearch/detectron2.git /tmp/pip-req-build-cmdt0cyg&#27;[0m  Resolved https://github.com/facebookresearch/detectron2.git to commit eddac5d5b4fce9b5eba5518f9ee348fa3a531407  Preparing metadata (setup.py): started  Preparing metadata (setup.py): finished with status 'error'&#27;[91m  error: subprocess-exited-with-error    × python setup.py egg_info did not run successfully.  │ exit code: 1  ╰─> [6 lines of output]      Traceback (most recent call last):        File "<string>", line 36, in <module>        File "<pip-setuptools-caller>", line 34, in <module>        File "/tmp/pip-req-build-cmdt0cyg/setup.py", line 10, in <module>          import torch      ModuleNotFoundError: No module named 'torch'      [end of output]    note: This error originates from a subprocess, and is likely not a problem with pip.&#27;[0m&#27;[91merror: metadata-generation-failed× Encountered error while generating package metadata.╰─> See above for output.note: This is an issue with the package mentioned above, not pip.hint: See above for details.&#27;[0m[notice] A new release of pip available: 22.1.2 -> 23.1.2[notice] To update, run: /opt/dataiku/code-env/bin/python -m pip install --upgrade pipRemoving intermediate container 4eaf662bfd5fThe command '/opt/dataiku/code-env/bin/python -m pip install --extra-index-url https://amrab@orsted.com:cmVmdGtuOjAxOjE3MTYwMjAyMDg6ZGRNMEVpVkdyNUM5QzRZY3NuQ2JKUkcxTlNj@orsted.jfrog.io/artifactory/incubator-pytorch-remote -r code-env/pip.packages.txt' returned a non-zero code: 1

Does anybody know how to solve this? it works without the container figurations...

---

Operating system used: Windows 10

Alexandru

Hi Simone,

From the lines, you shared :
line 10, in <module> import torch ModuleNotFoundError: No module named 'torch'

It's failing when trying to install git+https://github.com/facebookresearch/detectron2.git
Changing this to https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.8/detectron2-0.5%2Bcu101-cp38-cp38-linux_x86_64.whl instead seems to work fine

torch==1.8.1
torchvision==0.9.1
pyyaml==5.1
opencv-python==4.6.0.66
dash==2.6.1
dash-bootstrap-components==1.2.1
https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.8/detectron2-0.5+cu101-cp38-cp38-linux_x86_64.whl

It appears this was broken upstream based on this threads so when this originally created it would have worked but no longer work : https://github.com/facebookresearch/detectron2/issues/4386