Environment creation failed (Package setuptools)

Options
BHL1
BHL1 Dataiku DSS Core Designer, Registered Posts: 6 ✭✭✭
edited July 16 in Setup & Configuration

Hello !

I'm on DSS 8.0 and trying to create a new env but i have this error :

ERROR: Package 'setuptools' requires a different Python: 3.6.9 not in '>=3.7'

Can any one have a solution please !

Thanks !

Tagged:
· Share on FacebookShare on Twitter

Answers

  • arnaudde
    arnaudde Dataiker Posts: 52 Dataiker
    Options

    Hello,

    What python version are you using ?

    DSS 8.0 is compatible with Python versions 2.7, 3.5 and 3.6

    see our code env documentation

    Best

    · Share on FacebookShare on Twitter
  • BHL1
    BHL1 Dataiku DSS Core Designer, Registered Posts: 6 ✭✭✭
    Options

    I'm using Python 3.6

    I

    · Share on FacebookShare on Twitter
  • Sergey
    Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
    edited July 17
    Options

    Hi @BHL1
    ,

    This error means that setuptools that is being installed in this code env is higher than 59.6.0 (as starting from this version, setuptools requires py3.7). For py36 code env DSS installs 59.6 as the latest available: 

    ./py36_community/bin/pip3 list | grep setuptools
setuptools       59.6.0

    Can you provide more details on your setup? What is OS type and version? How this python 3.6 was installed on this host?

    · Share on FacebookShare on Twitter
  • BHL1
    BHL1 Dataiku DSS Core Designer, Registered Posts: 6 ✭✭✭
    Options

    Thank you for your quick answer.

    I'm using redhat VM v6.

    Is there any other option besides migrating to higher version or manually installing the necessary libraries to create a python env ?

    · Share on FacebookShare on Twitter
  • Sergey
    Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
    Options

    Hi @BHL1

    RHEL6 is out of support by many software vendors. I wasn't even able to install the basic packages required by DSS on this OS to find the possible solution/explanation for this case.

    As far as I remember we didn't support this OS in DSS8 as well.

    So you might need to upgrade the OS and try to create the code env later on.

    · Share on FacebookShare on Twitter
  • BHL1
    BHL1 Dataiku DSS Core Designer, Registered Posts: 6 ✭✭✭
    Options

    Hello Dataiker,

    I've cheked ther version of RHEL, it was v7.9 and not v6.

    Sorry for the mix-up

    Thanks,

    Best

    · Share on FacebookShare on Twitter
  • Sergey
    Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
    Options

    Hi @BHL1

    Can you please create a new py36 code env (reproduce the issue) and attach createPythonEnv.log (found on code env -> logs tab)? If this file is not being created -> attach backend.log (administration->maintenance->log files). The log can be quite big (>10mb), so it is better to upload it to dl.dataiku.com. Please send us the generated link in this case.

    · Share on FacebookShare on Twitter
  • BHL1
    BHL1 Dataiku DSS Core Designer, Registered Posts: 6 ✭✭✭
    Options

    Hey !

    Here is the error log and a screenshot

    Thank you for your help

    LOGS :

    {"severity":"INFO","logger":"dku.audit.generic","topic":"generic","message":{"callPath":"/api/code-envs/design/create","auditTopic":"generic","msgType":"admin-code-env-create","authSource":"USER_FROM_UI","envLang":"PYTHON","envName":"Test_py","clientIP":"***.*.*.*","authUser":"dss*****","originalIP":"**.***.***.***"},"mdc":{"apiCall":"/api/code-env/design/create","user":"dss*****"},"callTime":3,"timestamp":"2022-02-17T09:34:44.750+0100"}
    {"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/code-envs/design/create","auditTopic":"apicall","authSource":"USER_FROM_UI","envLang":"PYTHON","clientIP":"***.*.*.*","time":7,"authUser":"dssd****","originalIP":"*********"},"mdc":{"apiCall":"/api/code-envs/design/create","user":"dss"},"callTime":7,"timestamp":"2022-02-17T09:34:44.754+0100"}
    {"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/futures/get-update","auditTopic":"apicall","authSource":"USER_FROM_UI","futureId":"peDY4S2W","clientIP":"********","time":1,"authUser":"dssd","originalIP":"*******"},"mdc":{"apiCall":"/api/futures/get-update","user":"dss"},"callTime":1,"timestamp":"2022-02-17T09:34:44.818+0100"}
    {"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/futures/get-update","auditTopic":"apicall","authSource":"USER_FROM_UI","futureId":"peDY4S2W","clientIP":"*********","time":1,"authUser":"dssd","originalIP":"*******"},"mdc":{"apiCall":"/api/futures/get-update","user":"dss"},"callTime":1,"timestamp":"2022-02-17T09:34:45.367+0100"}
    {"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/futures/get-update","auditTopic":"apicall","authSource":"USER_FROM_UI","futureId":"peDY4S2W","clientIP":"*********","time":1,"authUser":"dssd","originalIP":"********"},"mdc":{"apiCall":"/api/futures/get-update","user":"dss"},"callTime":1,"timestamp":"2022-02-17T09:34:45.920+0100"}
    {"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/futures/get-update","auditTopic":"apicall","authSource":"USER_FROM_UI","futureId":"peDY4S2W","clientIP":"*******","time":1,"authUser":"dss","originalIP":"********"},"mdc":{"apiCall":"/api/futures/get-update","user":"dss"},"callTime":1,"timestamp":"2022-02-17T09:34:46.470+0100"}
    {"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/futures/get-update","auditTopic":"apicall","authSource":"USER_FROM_UI","futureId":"peDY4S2W","clientIP":"*********","time":1,"authUser":"dssd","originalIP":"********"},"mdc":{"apiCall":"/api/futures/get-update","user":"dss"},"callTime":1,"timestamp":"2022-02-17T09:34:47.025+0100"}

    Screenshot : attached

    · Share on FacebookShare on Twitter
  • Sergey
    Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
    Options

    Hi @BHL1

    Thanks, but audit.log doesn't give any valuable details on the root cause of the issue.

    We will need a backend.log (administration->maintenance->log files) and createPythonEnv.log from the code env. If the latter one is not available on the UI, you can get it from <DSS_DATA_DIR>/code-envs/logs/python/<CODE_ENV_NAME> directory.

    · Share on FacebookShare on Twitter
  • BHL1
    BHL1 Dataiku DSS Core Designer, Registered Posts: 6 ✭✭✭
    edited July 17
    Options

    Hey !

    Here is an extract of the backend log.

    [2022/02/17-20:00:00.000] [DefaultQuartzScheduler_QuartzSchedulerThread] [DEBUG] [org.quartz.simpl.PropertySettingJobFactory]  - Producing instance of Job 'reporting.send-user-offline-queues', class=com.dataiku.dip.maintainance.SendUserOfflineQueuesTask
[2022/02/17-20:00:00.001] [DefaultQuartzScheduler_Worker-4] [DEBUG] [org.quartz.core.JobRunShell]  - Calling execute on job reporting.send-user-offline-queues
[2022/02/17-20:00:00.001] [DefaultQuartzScheduler_Worker-4] [INFO] [dku.notifications.offline]  - Start sending user offline queue
[2022/02/17-20:00:00.011] [DefaultQuartzScheduler_Worker-4] [INFO] [dku.notifications.offline]  - No offline queue messages to send
[2022/02/17-20:00:00.011] [DefaultQuartzScheduler_QuartzSchedulerThread] [DEBUG] [org.quartz.simpl.PropertySettingJobFactory]  - Producing instance of Job 'task-execution.kill-timedout-tasks', class=com.dataiku.dip.scheduler.runnables.StoppableWithTimeoutService$KillExpiredCallables
[2022/02/17-20:00:00.012] [DefaultQuartzScheduler_Worker-7] [DEBUG] [org.quartz.core.JobRunShell]  - Calling execute on job task-execution.kill-timedout-tasks
[2022/02/17-20:00:00.012] [DefaultQuartzScheduler_Worker-7] [INFO] [com.dataiku.dip.scheduler.runnables.StoppableWithTimeoutService]  - Checking timeout for 0 processes
[2022/02/17-20:00:00.444] [qtp*********] [DEBUG] [dku.tracing]  - [ct: 0] Start call: /api/admin/logs/list [GET] user=dss
[2022/02/17-20:00:00.454] [qtp*******42] [DEBUG] [dku.tracing]  - [ct: 10] Done call: /api/admin/logs/list [GET] time=10ms user=dss
[2022/02/17-20:00:00.542] [qtp*********9] [DEBUG] [dku.tracing]  - [ct: 1] Start call: /api/admin/diagnostics/get-latest-diagnosis [GET] user=dss
[2022/02/17-20:00:00.542] [qtp*********9] [DEBUG] [dku.tracing]  - [ct: 1] Done call: /api/admin/diagnostics/get-latest-diagnosis [GET] time=1ms user=dss

[2022/02/17-20:00:04.436] [q58] [DEBUG] [dku.tracing]  - [ct: 1] Start call: /api/admin/logs/get-content [GET] user=dss [name=backend.log]
    · Share on FacebookShare on Twitter
  • Sergey
    Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
    Options

    Hi @BHL1

    We have recently seen the same issue. It was caused by the custom PyPI artifactory where metadata was broken allowing setuptools to be installed with unsupported python versions.

    To add more context, every package inside any artifactory during indexing takes the "requires_python" field. That's true, that pip during package installation (if no version is specified), takes the latest available BUT prior to this, pip "asks" repository (based on the metadata) what version is the max available for this particular python version the installation is performed.
    Artifactory should provide ONLY the ones that suit this python version while here artifactory "was saying" that all the packages are available for python3.6 (including the ones that are not).
    Are you using custom PyPi artifactory?
    · Share on FacebookShare on Twitter
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…