Can't create Conda code envs

UserBird
UserBird Dataiker, Alpha Tester Posts: 535 Dataiker

Hi, I am trying to create a code environment managed by conda from the settings interface but it keeps failing and returning the following message:

Environment creation failed ERR_CODEENV_ENV_CREATION_FAILED

Cannot run program "conda": error=2, No such file or directory, caused by: IOException: error=2, No such file or directory

My laptop is a Macbook Pro running High Sierra with Anaconda is installed in the default directory (/users/xxx/anaconda3) and conda runs fine from the command line. Python 3.6 is installed and included on the PATH variable.

Only creating managed python 2.7 code envs seems to work as pyenv is unable to find python3 even when its linked in /usr/local/bin from [..]/Library/Python.framework/ and works perfectly when invoked from the command line and other apps. This is the message in the backend log:

  • Environment creation failed

    Process execution failed (/Users/deccar/Library/DataScienceStudio/dss_home/bin/python) (return code 3)

The path python3.6 (from --python=python3.6) does not exist



I haven't managed to import existing conda envs either. Can you explain how should I configure the system to make it work?

Thanks in advance

Answers

  • matthias_funke
    matthias_funke Dataiker Alumni Posts: 12 ✭✭✭✭✭
    Hi Deccar, the doc (https://doc.dataiku.com/dss/latest/code-envs/conda.html) says conda should be in your DSS path. DSS manages it own environments, and even if you tell it to use conda it will manage a separate set of folders (separate from your "root" environment that you can use outside of DSS).

    Hope this helps!
  • UserBird
    UserBird Dataiker, Alpha Tester Posts: 535 Dataiker
    Hi Matthias,

    Yeah, the documentation says to include conda in your DSS path, but I haven't managed to find the way to do it. Conda's bin directory is in my $PATH but doesn't seem to do the trick.
  • tgb417
    tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,598 Neuron
    I have the same questions about adding using anaconda in DSS.
    1. Which path do I need? (How do I determine the correct path that I will need.)
    2. Where should this path be added to the DSS configuration?
  • Wuser92
    Wuser92 Registered Posts: 20 ✭✭✭✭
    Anyone found a solution for this? Having the same issue...
  • tgb417
    tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,598 Neuron
    I've made some progress.

    in my ~/.bash_profile I added the following line very late in the file. But before the conda.sh line.

    export PATH="/Users/[My User Home Directory]/anaconda/bin:$PATH"

    Then in my ~/Library/DataScienceStudio/dss_home/bin/env-site.sh

    I added:

    export PATH=/Users/[My User Home Directory]/anaconda/bin:$PATH

    To get R to work with the latest version of R at this time. (Version 3.4) I also added the following to the env-site.sh file.

    export DKURBIN=/Library/Frameworks/R.framework/Versions/3.4/Resources/bin/R

    You should check for the existence of this directory before putting this path in env-site.sh

    Note [My User Home Directory ] should be replaced with your user home directory.

    I have not gotten Anaconda Navigator based code envs to work when they are created in Anaconda Navigator first. These environments were created by letting DSS use Conda to set up the environments. Once set up I seem to be able to make updates to the environments with Anaconda Navigator. However, I'm not at all clear if that is a good idea.

    This took me days to figure out... I wish this was a bit easier.
  • Wuser92
    Wuser92 Registered Posts: 20 ✭✭✭✭
    Thanks for your reply!

    One note: I don't think you need the first step (changing your ~/.bash_profile) since the issue originally is that DSS can't find the binaries for conda / python, which is solved by adding the local conda directory to the $PATH in the second step.

    So if anyone else is having these issues, I think adding:
    export PATH=$PATH:/Users/[username]/anaconda/bin
    to the ~/Library/DataScienceStudio/dss_home/bin/env-site.sh should work fine

    THANKS!
  • Karanshade
    Karanshade Registered Posts: 2 ✭✭✭✭
    export PATH=$PATH:/Users/[username]/anaconda/bin
    to the ~/Library/DataScienceStudio/dss_home/bin/env-site.sh should work fine
    ^^^^^^^^^^^^^^^^^^^^^^^^
    It does indeed, for anyone having this issue, this is the simplest solution.
Setup Info
    Tags
      Help me…