Discover this year's submissions to the Dataiku Frontrunner Awards and give kudos to your favorite use cases and success stories!READ MORE

Setting Up Python 3.7 and Anaconda Code Env

Solved!
mahmoud_shihab
Level 3
Level 3
Setting Up Python 3.7 and Anaconda Code Env

Hi All,

I am having some trouble setting Dataiku code environments with any Python other than 2.7 & 3.6.

Further more, I can't seem to get the "Use Conda" option either.
I know anaconda is considered a tier 2 support item, but it's required for my current project.

Does anyone know how to get these features working?

Thank you in advance!


Operating system used: Red Hat Enterprise Linux


Operating system used: Red Hat Enterprise Linux 8.5

1 Solution
sergeyd
Dataiker
Dataiker

Hi, 

Thanks. So PATH is not the same seen from the CLI so obviously, conda cannot be found. You can force DSS to use modified PATH by editing DATA_DIR/bin/env-site.sh file and restarting DSS: 

export PATH=/<PATH_TO_CONDA_BIN_DIR>/:$PATH

 

View solution in original post

15 Replies
sergeyd
Dataiker
Dataiker

Hi @mahmoud_shihab 

Can you please post a screenshot with the error/warning you have while creating conda code env so we can understand the issue?  

0 Kudos
mahmoud_shihab
Level 3
Level 3
Author

Hi @sergeyd,

Of course.

Sorry about that. I thought I uploaded the pictures.

0 Kudos
sergeyd
Dataiker
Dataiker

Hi @mahmoud_shihab 

Thanks for the screenshots. It's not about Tier2 support. It's due to the fact that conda physically is not installed on the DSS server (or cannot be found from the PATH env variable). The same applies to the Python3.7 binary. 

You will need to install conda and make sure it's available in DSS system user's PATH. If you are planning to use Python3.7 natively installed (not via conda) you will need to install it as well. Your sysadmins should help you with this task. 

0 Kudos
mahmoud_shihab
Level 3
Level 3
Author

Hi @sergeyd,

I currently have anaconda installed on the linux machine and available to DSS (as it is the only user and considered an sudoer).

But it still can't be found...

Do I need to install it in a specific way (for both anaconda and any python binary I need)?

0 Kudos
sergeyd
Dataiker
Dataiker

Hi,

Thanks. If dss is a DSS system user, make sure that you have restarted DSS after applying changes to the user profile (including PATH changes). 

0 Kudos
mahmoud_shihab
Level 3
Level 3
Author

Hi @sergeyd,

Everything I have showed you is after installing Anaconda and restarting both DSS and the Linux system.

I'm not sure why it's not working.

I feel like I am missing something.

It might be worth mentioning that Dataiku is installed on a drive mounted on /opt/dataiku

Do I need to install the python and anaconda binaries there?

0 Kudos
sergeyd
Dataiker
Dataiker

Hi, 

You do not need to install conda into DSS mounted volume. What you are getting while running the next commands by DSS system user: 

which conda
conda info

?

0 Kudos
mahmoud_shihab
Level 3
Level 3
Author

Here is the info you asked about @sergeyd 

0 Kudos
sergeyd
Dataiker
Dataiker

Hi,

Ok, thanks. So from CLI all looks good. Now, let's check what DSS "sees". If you do not have UIF enabled, please open python notebook and run the next in the cell: 

!id
print("-------")
!echo $PATH
print("-------")
!which conda

 

0 Kudos
mahmoud_shihab
Level 3
Level 3
Author

Hi @sergeyd 

Here is the result of the commands

0 Kudos
sergeyd
Dataiker
Dataiker

Hi, 

Thanks. So PATH is not the same seen from the CLI so obviously, conda cannot be found. You can force DSS to use modified PATH by editing DATA_DIR/bin/env-site.sh file and restarting DSS: 

export PATH=/<PATH_TO_CONDA_BIN_DIR>/:$PATH

 

mahmoud_shihab
Level 3
Level 3
Author

@sergeyd 

Like This?

After This, do I restart DSS?

sergeyd
Dataiker
Dataiker

Yes, that's fine. Restart of DSS is required as I mentioned earlier. 

0 Kudos
mahmoud_shihab
Level 3
Level 3
Author

@sergeyd 

It seems to be working!

Thank you!

0 Kudos
sergeyd
Dataiker
Dataiker

Hi @mahmoud_shihab 

Glad to hear this. Please mark this as solved then so others can quickly find the solution in the future.