Upgrading DSS for Ubuntu 18.04: fails with python import error "cannot import name _remove_dead_weak

UserBird

Hi,

I upgraded ubuntu to 18.04, after reading that dss is compatible with it since version 4.2.2.

Nevertheless, when I try to install dss 4.2.2, it fails with this error:



*********************************************
* Dataiku DSS installer *
*********************************************
[+] /home/inactinique/dataiku-data already exists and is a DSS installation
[+] Saving installation log to /home/inactinique/dataiku-data/run/install.log
[+] Using Java at /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java : openjdk version "1.8.0_162"
[+] Checking required dependencies
+ Detected OS distribution : ubuntu 18.04
+ Checking required packages...
[+] Migrating Python environment
[+] Precompiling Dataiku Python code
[+] Precompiling Jupyter Python code
[+] Precompiling third-party Python code
[+] Migrating data directory
Traceback (most recent call last):
File "/opt/dataiku-dss-4.2.2/scripts/dkuinstall/migrate_auto.py", line 5, in
import migration_base, migrate_dss, migrate_apinode
File "/opt/dataiku-dss-4.2.2/scripts/dkuinstall/migration_base.py", line 7, in
import json, logging
File "/usr/lib/python2.7/logging/__init__.py", line 26, in
import sys, os, time, cStringIO, traceback, warnings, weakref, collections
File "/usr/lib/python2.7/weakref.py", line 14, in
from _weakref import (
ImportError: cannot import name _remove_dead_weakref



Has someone any idea about what's going wrong?

Best,

Frédéric

pbertin

Hi Frédéric,

this is a known limitation of virtualenv with respect to an upgrade of the underlying Python (this has already been seen to happen on previous upgrades of Ubuntu distributions) : the Python virtual environment within the DSS data dir still contains references to the pre-upgrade Python, and breaks with low-level errors on startup.

The solution is most probably to fully rebuild the Python environment used by DSS, as described here:
https://doc.dataiku.com/dss/latest/installation/migrations.html#migrating-the-data-directory

- move away (rename) the DATADIR/pyenv directory
- re-run the DSS installer in upgrade mode : this should rebuild a new virtualenv
- optionally, reinstall any locally-installed package (you can look for them by listing the lib/python2.7/site-packages in the old virtualenv)

Note that any DSS-managed Python code environments will probably have to be rebuilt as well (this can be done from the DSS UI)

Regards
Patrice Bertin
Dataiku

UserBird

Dear Patrice,
it just perfectly worked.
Merci!