Skip to content
  • Register

    • Setting Up Python 3.7 and Anaconda Code Env

    Added on April 19, 2022 10:51AM

    Likes: 1

    Replies: 16

    Tags
    Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner
    in Setup & Configuration

    Hi All,

    I am having some trouble setting Dataiku code environments with any Python other than 2.7 & 3.6.

    Further more, I can't seem to get the "Use Conda" option either.
    I know anaconda is considered a tier 2 support item, but it's required for my current project.

    Does anyone know how to get these features working?

    Thank you in advance!

    Operating system used: Red Hat Enterprise Linux

    Operating system used: Red Hat Enterprise Linux 8.5

    · Share on FacebookShare on Twitter

    Welcome!

    It looks like you're new here. Sign in or register to get started.
    Sign In
    Register

    Best Answer

    • Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Registered Posts: 365 Dataiker
      edited July 2024 Answer ✓

      Hi,

      Thanks. So PATH is not the same seen from the CLI so obviously, conda cannot be found. You can force DSS to use modified PATH by editing DATA_DIR/bin/env-site.sh file and restarting DSS: 

      export PATH=/<PATH_TO_CONDA_BIN_DIR>/:$PATH

      · Share on FacebookShare on Twitter

    Answers

    • Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Registered Posts: 365 Dataiker

      Hi @mahmoud_shihab

      Can you please post a screenshot with the error/warning you have while creating conda code env so we can understand the issue?

      · Share on FacebookShare on Twitter
    • Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner

      Hi @sergeyd,

      Of course.

      Sorry about that. I thought I uploaded the pictures.

      · Share on FacebookShare on Twitter
    • Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Registered Posts: 365 Dataiker

      Hi @mahmoud_shihab

      Thanks for the screenshots. It's not about Tier2 support. It's due to the fact that conda physically is not installed on the DSS server (or cannot be found from the PATH env variable). The same applies to the Python3.7 binary.

      You will need to install conda and make sure it's available in DSS system user's PATH. If you are planning to use Python3.7 natively installed (not via conda) you will need to install it as well. Your sysadmins should help you with this task.

      · Share on FacebookShare on Twitter
    • Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner

      Hi @sergeyd
      ,

      I currently have anaconda installed on the linux machine and available to DSS (as it is the only user and considered an sudoer).

      But it still can't be found...

      Do I need to install it in a specific way (for both anaconda and any python binary I need)?

      · Share on FacebookShare on Twitter
    • Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Registered Posts: 365 Dataiker

      Hi,

      Thanks. If dss is a DSS system user, make sure that you have restarted DSS after applying changes to the user profile (including PATH changes).

      · Share on FacebookShare on Twitter
    • Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner

      Hi @sergeyd
      ,

      Everything I have showed you is after installing Anaconda and restarting both DSS and the Linux system.

      I'm not sure why it's not working.

      I feel like I am missing something.

      It might be worth mentioning that Dataiku is installed on a drive mounted on /opt/dataiku

      Do I need to install the python and anaconda binaries there?

      · Share on FacebookShare on Twitter
    • Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Registered Posts: 365 Dataiker
      edited July 2024

      Hi,

      You do not need to install conda into DSS mounted volume. What you are getting while running the next commands by DSS system user: 

      which conda
conda info

      ?

      · Share on FacebookShare on Twitter
    • Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner

      Here is the info you asked about @sergeyd

      · Share on FacebookShare on Twitter
    • Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Registered Posts: 365 Dataiker
      edited July 2024

      Hi,

      Ok, thanks. So from CLI all looks good. Now, let's check what DSS "sees". If you do not have UIF enabled, please open python notebook and run the next in the cell: 

      !id
print("-------")
!echo $PATH
print("-------")
!which conda

      · Share on FacebookShare on Twitter
    • Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner

      Hi @sergeyd

      Here is the result of the commands

      · Share on FacebookShare on Twitter
    • Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner

      @sergeyd

      Like This?

      After This, do I restart DSS?

      · Share on FacebookShare on Twitter
    • Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Registered Posts: 365 Dataiker

      Yes, that's fine. Restart of DSS is required as I mentioned earlier.

      · Share on FacebookShare on Twitter
    • Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner

      @sergeyd

      It seems to be working!

      Thank you!

      · Share on FacebookShare on Twitter
    • Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Registered Posts: 365 Dataiker

      Hi @mahmoud_shihab

      Glad to hear this. Please mark this as solved then so others can quickly find the solution in the future.

      · Share on FacebookShare on Twitter
    • Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 6 Partner

      Hi @sergeyd
      , how are you?

      I am trying to follow the same steps you have suggested for Mahmoud but for the windows version.

      I have found the conda path like follows:

      conda path.png

      And I have found the env-site.ps1 file (with the path: C:\Users\LENOVO\AppData\Local\Dataiku\DataScienceStudio\dss_home\bin\env-site.ps1), and inside the file I used the following:

      setx PATH C:\Users\LENOVO\anaconda3;C:\Users\LENOVO\anaconda3\Library\mingw-w64\bin;C:\Users\LENOVO\anaconda3\Library\usr\bin;C:\Users\LENOVO\anaconda3\Library\bin;C:\Users\LENOVO\anaconda3\Scripts;C:\Users\LENOVO\anaconda3\bin;C:\Users\LENOVO\anaconda3\condabin;
      But I still recieve this error:
      env-error.png
      Would you kindly help me with this?
      · Share on FacebookShare on Twitter

    Welcome!

    It looks like you're new here. Sign in or register to get started.
    Sign In
    Register
    Bookmark this topic

    Welcome!

    It looks like you're new here. Sign in or register to get started.
    Sign In
    Register

    Quick Links

    Categories

    Help me…
    …ask a question…search for an answer…learn to use the Community…sign in

    This website uses cookies. By clicking OK, you consent to the use of cookies. Read our cookie policy.

    AcceptReject
    Top