Can't create Conda code envs
Hi, I am trying to create a code environment managed by conda from the settings interface but it keeps failing and returning the following message:
Environment creation failed ERR_CODEENV_ENV_CREATION_FAILED
Cannot run program "conda": error=2, No such file or directory, caused by: IOException: error=2, No such file or directory
My laptop is a Macbook Pro running High Sierra with Anaconda is installed in the default directory (/users/xxx/anaconda3) and conda runs fine from the command line. Python 3.6 is installed and included on the PATH variable.
Only creating managed python 2.7 code envs seems to work as pyenv is unable to find python3 even when its linked in /usr/local/bin from [..]/Library/Python.framework/ and works perfectly when invoked from the command line and other apps. This is the message in the backend log:
-
Environment creation failed
Process execution failed (/Users/deccar/Library/DataScienceStudio/dss_home/bin/python) (return code 3)
The path python3.6 (from --python=python3.6) does not exist
I haven't managed to import existing conda envs either. Can you explain how should I configure the system to make it work?
Thanks in advance
Answers
-
Hi Deccar, the doc (https://doc.dataiku.com/dss/latest/code-envs/conda.html) says conda should be in your DSS path. DSS manages it own environments, and even if you tell it to use conda it will manage a separate set of folders (separate from your "root" environment that you can use outside of DSS).
Hope this helps! -
Hi Matthias,
Yeah, the documentation says to include conda in your DSS path, but I haven't managed to find the way to do it. Conda's bin directory is in my $PATH but doesn't seem to do the trick. -
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,601 NeuronI have the same questions about adding using anaconda in DSS.
1. Which path do I need? (How do I determine the correct path that I will need.)
2. Where should this path be added to the DSS configuration? -
Anyone found a solution for this? Having the same issue...
-
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,601 NeuronI've made some progress.
in my ~/.bash_profile I added the following line very late in the file. But before the conda.sh line.
export PATH="/Users/[My User Home Directory]/anaconda/bin:$PATH"
Then in my ~/Library/DataScienceStudio/dss_home/bin/env-site.sh
I added:
export PATH=/Users/[My User Home Directory]/anaconda/bin:$PATH
To get R to work with the latest version of R at this time. (Version 3.4) I also added the following to the env-site.sh file.
export DKURBIN=/Library/Frameworks/R.framework/Versions/3.4/Resources/bin/R
You should check for the existence of this directory before putting this path in env-site.sh
Note [My User Home Directory ] should be replaced with your user home directory.
I have not gotten Anaconda Navigator based code envs to work when they are created in Anaconda Navigator first. These environments were created by letting DSS use Conda to set up the environments. Once set up I seem to be able to make updates to the environments with Anaconda Navigator. However, I'm not at all clear if that is a good idea.
This took me days to figure out... I wish this was a bit easier. -
Thanks for your reply!
One note: I don't think you need the first step (changing your ~/.bash_profile) since the issue originally is that DSS can't find the binaries for conda / python, which is solved by adding the local conda directory to the $PATH in the second step.
So if anyone else is having these issues, I think adding:
export PATH=$PATH:/Users/[username]/anaconda/bin
to the ~/Library/DataScienceStudio/dss_home/bin/env-site.sh should work fine
THANKS! -
export PATH=$PATH:/Users/[username]/anaconda/bin
to the ~/Library/DataScienceStudio/dss_home/bin/env-site.sh should work fine
^^^^^^^^^^^^^^^^^^^^^^^^
It does indeed, for anyone having this issue, this is the simplest solution.