Are there different ways to set up code environments?

Graham_E

I am trying to install pytorch in python3 in a code environment in data science studio. I can install it in the python3.5 install on the system that Data Science Studio is installed on. I've tried putting torch in the REQUESTED PACKAGES (PIP) part of the code environment administration but that doesn't work because pytorch can't be installed directly through pip. Pytorch requires you to download a pip wheel for installing it (instructions are here: http://pytorch.org/).

I've noticed there is a TODO section coming in the documentation (https://doc.dataiku.com/dss/latest/code-envs/operations-python.html) which probably gives more details about this. I will also be curious how to install a custom project (from git) in a way that it can be used in a custom code environment. FYI I have been able to install both pytorch and the custom project I am alluding to for use with the system install of python3.5 on the server I have DSS installed on.

I'm not sure if my description of the problem is clear so please ask questions if there are any ambiguities. Thanks!

Clément_Stenac

Hi,

The syntax in the "requested packages" is actually a pip requirements file, so you can simply put the URL of your wheel here

So in essence, you just need the following in your "requested packages"

http://download.pytorch.org/whl/cu80/torch-0.2.0.post3-cp35-cp35m-manylinux1_x86_64.whl
torchvision

Graham_E

Awesome, thanks! I'm also curious about how to install a package that isn't done through pip. In the docs you have:

TODO: Explain that admin first downloads them and then how to put that in requirements.txt

I'm trying to install this if it helps: https://github.com/huggingface/torchMoji. The install command I use is:

pip3 install -e .

Clément_Stenac

Clone the repository on the DSS machine as you would normally do, then add the following line to the "Requested packages":

-e /absolute/path/to/where/you/cloned/torchMoji

MarkPundurs

pytorch can't be installed directly through pip. Pytorch requires you to download a pip wheel for installing it

---

As I understand it, this is incorrect as it stands; Pytorch requires you to download a pip wheel for installing it with CUDA support.

And in 12.1 it may be the case that no special handling is required; I simply entered torch in the "Requested packages" list and had the following packages (among others) installed:

nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105