Best Practices for Updating Python

[Jason]

So as background, I'm not really a Linux server administrator, and our Linux administrators aren't really dataiku administrators (also, they're not really python people per-se).  We have an on-prem install.

With that background, we've got python 3.6 and 3.7 on our instances, and we'd like to add each major point version up to 3.10 or so (our Dataiku version number doesn't support 3.11).

My question is: How does Dataiku know which versions of python are available for environments to use?  I see version numbers in the pulldown despite the fact that they are not installed on the server, and poking around, I can see a few versions installed, but I don't know what the recommended method is for installing python in my environment (Red Hat).  Can I just navigate to the dataiku_data/pyenv/bin directory and install with PIP?  If I do that do I need to do something to tell Dataiku where to find my new version?  I'm hoping it's just automagic, but I'm hesitant to give such vague instructions to my server admin.  I've been googling it for a few hours now, and I'm not finding any documentation that is really specific (leading me to believe in the magic part).

Help?

---

Operating system used: Red Hat

[Turribeach · Accepted solution]

Hi, even though you see the options listed on the dropdown menu when you try to create a new Python code environment each option will not work unless the corresponding Python version is actually installed in the system. 

Can I just navigate to the dataiku_data/pyenv/bin directory and install with PIP?

>> No! This is the built-in Python environment and you should stay away from it! I recently wrote a Product Idea which covers this in detail so have a good read.

To install more Python versions in RHEL you simply run "yum install package_name" and it will be installed. Then reboot your system and voilà, you can create code environments in the new Python version. The RHEL package names are "python38", "python39" and "python3.11" (*). Sometimes the devel packages are needed so I install those too (ie python38-devel). How does Dataiku know you did that? Because the Python binaries get added to /usr/bin/ which is in the path so Dataiku can see them.

(*) RHEL has decided to skip Python 3.10 so you can only install 3.8, 3.9 and 3.11 (if you are in Dataiku v12) from RHEL repos. If you want 3.10 you will need to go to Python.org and download the source and do a manual install or get another package from another repository but this won't be curated by RHEL so there is some risk attached to that.

[Jason]

Thank you!  This is exactly what I was looking for!