Environment creation failed (Package setuptools)

BHL1

Hello !

I'm on DSS 8.0 and trying to create a new env but i have this error :

ERROR: Package 'setuptools' requires a different Python: 3.6.9 not in '>=3.7'

Can any one have a solution please !

Thanks !

arnaudde

Hello,

What python version are you using ?

DSS 8.0 is compatible with Python versions 2.7, 3.5 and 3.6

see our code env documentation

Best

BHL1

I'm using Python 3.6

I

Sergey

Hi @BHL1
,

This error means that setuptools that is being installed in this code env is higher than 59.6.0 (as starting from this version, setuptools requires py3.7). For py36 code env DSS installs 59.6 as the latest available:

./py36_community/bin/pip3 list | grep setuptools
setuptools       59.6.0

Can you provide more details on your setup? What is OS type and version? How this python 3.6 was installed on this host?

BHL1

Thank you for your quick answer.

I'm using redhat VM v6.

Is there any other option besides migrating to higher version or manually installing the necessary libraries to create a python env ?

Sergey

Hi @BHL1

RHEL6 is out of support by many software vendors. I wasn't even able to install the basic packages required by DSS on this OS to find the possible solution/explanation for this case.

As far as I remember we didn't support this OS in DSS8 as well.

So you might need to upgrade the OS and try to create the code env later on.

BHL1

Hello Dataiker,

I've cheked ther version of RHEL, it was v7.9 and not v6.

Sorry for the mix-up

Thanks,

Best

Sergey

Hi @BHL1

Can you please create a new py36 code env (reproduce the issue) and attach createPythonEnv.log (found on code env -> logs tab)? If this file is not being created -> attach backend.log (administration->maintenance->log files). The log can be quite big (>10mb), so it is better to upload it to dl.dataiku.com. Please send us the generated link in this case.

BHL1

Hey !

Here is the error log and a screenshot

Thank you for your help

LOGS :

{"severity":"INFO","logger":"dku.audit.generic","topic":"generic","message":{"callPath":"/api/code-envs/design/create","auditTopic":"generic","msgType":"admin-code-env-create","authSource":"USER_FROM_UI","envLang":"PYTHON","envName":"Test_py","clientIP":"***.*.*.*","authUser":"dss*****","originalIP":"**.***.***.***"},"mdc":{"apiCall":"/api/code-env/design/create","user":"dss*****"},"callTime":3,"timestamp":"2022-02-17T09:34:44.750+0100"}
{"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/code-envs/design/create","auditTopic":"apicall","authSource":"USER_FROM_UI","envLang":"PYTHON","clientIP":"***.*.*.*","time":7,"authUser":"dssd****","originalIP":"*********"},"mdc":{"apiCall":"/api/code-envs/design/create","user":"dss"},"callTime":7,"timestamp":"2022-02-17T09:34:44.754+0100"}
{"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/futures/get-update","auditTopic":"apicall","authSource":"USER_FROM_UI","futureId":"peDY4S2W","clientIP":"********","time":1,"authUser":"dssd","originalIP":"*******"},"mdc":{"apiCall":"/api/futures/get-update","user":"dss"},"callTime":1,"timestamp":"2022-02-17T09:34:44.818+0100"}
{"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/futures/get-update","auditTopic":"apicall","authSource":"USER_FROM_UI","futureId":"peDY4S2W","clientIP":"*********","time":1,"authUser":"dssd","originalIP":"*******"},"mdc":{"apiCall":"/api/futures/get-update","user":"dss"},"callTime":1,"timestamp":"2022-02-17T09:34:45.367+0100"}
{"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/futures/get-update","auditTopic":"apicall","authSource":"USER_FROM_UI","futureId":"peDY4S2W","clientIP":"*********","time":1,"authUser":"dssd","originalIP":"********"},"mdc":{"apiCall":"/api/futures/get-update","user":"dss"},"callTime":1,"timestamp":"2022-02-17T09:34:45.920+0100"}
{"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/futures/get-update","auditTopic":"apicall","authSource":"USER_FROM_UI","futureId":"peDY4S2W","clientIP":"*******","time":1,"authUser":"dss","originalIP":"********"},"mdc":{"apiCall":"/api/futures/get-update","user":"dss"},"callTime":1,"timestamp":"2022-02-17T09:34:46.470+0100"}
{"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/futures/get-update","auditTopic":"apicall","authSource":"USER_FROM_UI","futureId":"peDY4S2W","clientIP":"*********","time":1,"authUser":"dssd","originalIP":"********"},"mdc":{"apiCall":"/api/futures/get-update","user":"dss"},"callTime":1,"timestamp":"2022-02-17T09:34:47.025+0100"}

Screenshot : attached

Sergey

Hi @BHL1

Thanks, but audit.log doesn't give any valuable details on the root cause of the issue.

We will need a backend.log (administration->maintenance->log files) and createPythonEnv.log from the code env. If the latter one is not available on the UI, you can get it from <DSS_DATA_DIR>/code-envs/logs/python/<CODE_ENV_NAME> directory.

BHL1

Hey !

Here is an extract of the backend log.

[2022/02/17-20:00:00.000] [DefaultQuartzScheduler_QuartzSchedulerThread] [DEBUG] [org.quartz.simpl.PropertySettingJobFactory]  - Producing instance of Job 'reporting.send-user-offline-queues', class=com.dataiku.dip.maintainance.SendUserOfflineQueuesTask
[2022/02/17-20:00:00.001] [DefaultQuartzScheduler_Worker-4] [DEBUG] [org.quartz.core.JobRunShell]  - Calling execute on job reporting.send-user-offline-queues
[2022/02/17-20:00:00.001] [DefaultQuartzScheduler_Worker-4] [INFO] [dku.notifications.offline]  - Start sending user offline queue
[2022/02/17-20:00:00.011] [DefaultQuartzScheduler_Worker-4] [INFO] [dku.notifications.offline]  - No offline queue messages to send
[2022/02/17-20:00:00.011] [DefaultQuartzScheduler_QuartzSchedulerThread] [DEBUG] [org.quartz.simpl.PropertySettingJobFactory]  - Producing instance of Job 'task-execution.kill-timedout-tasks', class=com.dataiku.dip.scheduler.runnables.StoppableWithTimeoutService$KillExpiredCallables
[2022/02/17-20:00:00.012] [DefaultQuartzScheduler_Worker-7] [DEBUG] [org.quartz.core.JobRunShell]  - Calling execute on job task-execution.kill-timedout-tasks
[2022/02/17-20:00:00.012] [DefaultQuartzScheduler_Worker-7] [INFO] [com.dataiku.dip.scheduler.runnables.StoppableWithTimeoutService]  - Checking timeout for 0 processes
[2022/02/17-20:00:00.444] [qtp*********] [DEBUG] [dku.tracing]  - [ct: 0] Start call: /api/admin/logs/list [GET] user=dss
[2022/02/17-20:00:00.454] [qtp*******42] [DEBUG] [dku.tracing]  - [ct: 10] Done call: /api/admin/logs/list [GET] time=10ms user=dss
[2022/02/17-20:00:00.542] [qtp*********9] [DEBUG] [dku.tracing]  - [ct: 1] Start call: /api/admin/diagnostics/get-latest-diagnosis [GET] user=dss
[2022/02/17-20:00:00.542] [qtp*********9] [DEBUG] [dku.tracing]  - [ct: 1] Done call: /api/admin/diagnostics/get-latest-diagnosis [GET] time=1ms user=dss

[2022/02/17-20:00:04.436] [q58] [DEBUG] [dku.tracing]  - [ct: 1] Start call: /api/admin/logs/get-content [GET] user=dss [name=backend.log]

Sergey

Hi @BHL1

We have recently seen the same issue. It was caused by the custom PyPI artifactory where metadata was broken allowing setuptools to be installed with unsupported python versions.

To add more context, every package inside any artifactory during indexing takes the "requires_python" field. That's true, that pip during package installation (if no version is specified), takes the latest available BUT prior to this, pip "asks" repository (based on the metadata) what version is the max available for this particular python version the installation is performed.

Artifactory should provide ONLY the ones that suit this python version while here artifactory "was saying" that all the packages are available for python3.6 (including the ones that are not).

Are you using custom PyPi artifactory?