Want to Stop Rebuilding "Expensive" Parts of your Flow? Explicit Builds are the Answer!READ MORE

Python 3.7 Code Env Creation Issue

Solved!
mahmoud_shihab
Level 3
Level 3
Python 3.7 Code Env Creation Issue

I have anaconda & py3.7 (altinstall) installed first

I then installed DSS to ensure the path of DSS sees both of them

I can install via conda

I seem to be have in issue when installing via the path

At the inital creation, it gives a warning (see picture py-jup-warn)

I can access the code env, but I can't install anything. (see picture py37_ERR)

What am I missing?

I checked the path using a notebook (see picture dss_path)


Operating system used: RHEL-8.5

0 Kudos
2 Solutions
sergeyd
Dataiker
Dataiker

Hi, 

Thank for the diag. From the logs: 

    File "/opt/dataiku/data/code-envs/python/py37/lib/python3.7/site-packages/setuptools/windows_support.py", line 2, in <module>
      import ctypes
    File "/usr/local/lib/python3.7/ctypes/__init__.py", line 7, in <module>
      from _ctypes import Union, Structure, Array
  ModuleNotFoundError: No module named '_ctypes'

As you are using Py3.7 that was compiled, you need to make sure that the next development headers are present on the system and recompile it again: 

openssl-devel bzip2-devel libffi-devel sqlite-devel xz-devel

 

View solution in original post

0 Kudos
mahmoud_shihab
Level 3
Level 3
Author

Hi @sergeyd 

Before installing Py3.7, Anaconda, and DSS, I had run the following commands:

sudo yum groupinstall "Development Tools"
sudo yum install openssl-devel zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel xz-devel

I think these cover all the headers you mentioned except libffi-devel.

I will try to recompile and get back to you on this...

View solution in original post

0 Kudos
7 Replies
sergeyd
Dataiker
Dataiker

Hi @mahmoud_shihab 

Please provide code env diag so we can review your setup: 

Code env diagnostic.png

0 Kudos
mahmoud_shihab
Level 3
Level 3
Author

Hi @sergeyd 

Here you go

0 Kudos
sergeyd
Dataiker
Dataiker

Hi, 

Thank for the diag. From the logs: 

    File "/opt/dataiku/data/code-envs/python/py37/lib/python3.7/site-packages/setuptools/windows_support.py", line 2, in <module>
      import ctypes
    File "/usr/local/lib/python3.7/ctypes/__init__.py", line 7, in <module>
      from _ctypes import Union, Structure, Array
  ModuleNotFoundError: No module named '_ctypes'

As you are using Py3.7 that was compiled, you need to make sure that the next development headers are present on the system and recompile it again: 

openssl-devel bzip2-devel libffi-devel sqlite-devel xz-devel

 

0 Kudos
mahmoud_shihab
Level 3
Level 3
Author

Hi @sergeyd 

Before installing Py3.7, Anaconda, and DSS, I had run the following commands:

sudo yum groupinstall "Development Tools"
sudo yum install openssl-devel zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel xz-devel

I think these cover all the headers you mentioned except libffi-devel.

I will try to recompile and get back to you on this...

0 Kudos
mahmoud_shihab
Level 3
Level 3
Author

Hi @sergeyd,

I just installed libffi-devel.

I went to the python folder, ran the following:

sudo ./configure --enable-optimization
sudo make altinstall

I then stopped and restarted DSS

It SEEMINGLY created the environment with no issues, but seems to have an issue when I tried to install anything else.

I tried to test using the "Add Packages -> Visual Machine Learning with Bayesian Search -> Update".

It gave the error in the attached picture.

For expediency, I also added the diagnostic info.

0 Kudos
sergeyd
Dataiker
Dataiker

Hi, 

You need to split two packages that you have in one line:

numpyscikit-optimize>=0.7,<0.8

It should be:

numpy
scikit-optimize>=0.7,<0.8
0 Kudos
mahmoud_shihab
Level 3
Level 3
Author

I didn't notice that they were stuck together.

After splitting it, it seems to have worked.

I think because I was missing one of the required packages, it wasn't working the first time

Thank you so much!