Do you know the History of Data Science? READ MORE

Name External Conda environment - how to use containerized execution ?

Solved!
rona
Level 3
Name External Conda environment - how to use containerized execution ?

We've created a 'Name External Conda environment' in DSS and we would like to use this code environment in a python recipe with a containerized execution.

How to setup this 'Name External Conda environment' with containerized execution as the option to define the container is not available from DSS screen like for the other code environments ?

Annie

0 Kudos
1 Solution
sergeyd
Dataiker
Dataiker

Hi @rona 

If we are talking about Python3.8, you can only make this code env build from a regular (non-conda) python binary by passing over the path to the custom python interpreter: 

Screenshot 2021-07-03 at 00.16.06.png

Conda options will be limited to Python2.7, 3.5, 3.6, and 3.7 (for DSS9). Please note you will not be able to create a containerized code env that was built with a custom python interpreter. 

View solution in original post

0 Kudos
6 Replies
sergeyd
Dataiker
Dataiker

Hi @rona 

It's not possible to use such environments in the containerized executions because DSS is not aware of what packages are actually installed in that code env as the user installs the packages manually outside of DSS. 

If you need to have a code env that you can use in containerized execution (and built with conda) you will need to create a managed code env with "Use conda" checkbox enabled: 

Screenshot 2021-07-01 at 21.58.37.png

You will be able to enable containerized execution for it:

Screenshot 2021-07-01 at 22.00.07.png

Hope you will find this information helpful. 

0 Kudos
rona
Level 3
Author

Thanks a lot for this feedback.

We tried to use the code environment building with conda and for the package rdkit we had to add manually the channel conda-forge.

conda activate ... dss/code-envs/python/dssconda-py37
conda config --append channels conda-forge

 

Please, do you know if there is another way to add this configuration directly with the GUI of DSS ?

When we try to use the containerization with this code environment, there is an issue by doing this action outside DSS.

Annie

 

0 Kudos
sergeyd
Dataiker
Dataiker

Hi @rona 

Yes, you can add an additional conda channel by going to managed code env -> General tab where you need to uncheck "Inherit global settings" and add two lines to the Extra options for 'conda install': 

Screenshot 2021-07-02 at 14.00.20.png

This will add a conda-forge channel so "rdkit" package will be available for the installation. 

rona
Level 3
Author

Thanks a lot ! it works fine this way both on local and with containerized execution.

Additional question : can we build code environment with conda option in DSS with python 3.8 ? (even it's not supported in case of issues). 

Annie

0 Kudos
sergeyd
Dataiker
Dataiker

Hi @rona 

If we are talking about Python3.8, you can only make this code env build from a regular (non-conda) python binary by passing over the path to the custom python interpreter: 

Screenshot 2021-07-03 at 00.16.06.png

Conda options will be limited to Python2.7, 3.5, 3.6, and 3.7 (for DSS9). Please note you will not be able to create a containerized code env that was built with a custom python interpreter. 

View solution in original post

0 Kudos
rona
Level 3
Author

Thanks a lot Sergey for this information. It's clear now.

Annie

0 Kudos
A banner prompting to get Dataiku DSS