Installation / DSS Start failure
![PGuti](https://us.v-cdn.net/6038231/uploads/lithium_attachments/n4554iF3A64DE6E30275A8.jpg)
Hello!
I had a quite old instance of DSS on my mac (I used it last time in late 2017).
I wanted to run it but I got an error when trying to launch it.
So I thought OK I will just download the latest version and upgrade it.
I did this and when I'm trying to start DSS I get the following error:
[Install] ****************************************** * MIGRATION FAILED ****************************************** * Attempting to rollback Backing up partially-migrated: dss-version.json -> /Users/pierregutierrez/Library/DataScienceStudio/dss_home/_broken_migration_backup_20211029-103623/dss-version.json [Install] Restoring backup: _pre_migration_backup_20211029-103623/dss-version.json -> dss-version.json [Install] Backing up partially-migrated: bin/env-default.sh -> /Users/pierregutierrez/Library/DataScienceStudio/dss_home/_broken_migration_backup_20211029-103623/bin__env-default.sh Restoring backup: _pre_migration_backup_20211029-103623/bin__env-default.sh -> bin/env-default.sh [Install] Backing up partially-migrated: install.ini -> /Users/pierregutierrez/Library/DataScienceStudio/dss_home/_broken_migration_backup_20211029-103623/install.ini [Install] Restoring backup: _pre_migration_backup_20211029-103623/install.ini -> install.ini Backing up partially-migrated: databases -> /Users/pierregutierrez/Library/DataScienceStudio/dss_home/_broken_migration_backup_20211029-103623/databases [Install] Restoring backup: _pre_migration_backup_20211029-103623/databases -> databases [Install] Backing up partially-migrated: config -> /Users/pierregutierrez/Library/DataScienceStudio/dss_home/_broken_migration_backup_20211029-103623/config [Install] Restoring backup: _pre_migration_backup_20211029-103623/config -> config Restore complete, removing marker file [Install] Traceback (most recent call last): File "/Applications/DataScienceStudio.app/Contents/Resources/kit/scripts/dkuinstall/migrate_auto.py", line 30, in <module> migrate_dss.migrate(sys.argv[1]) File "/Applications/DataScienceStudio.app/Contents/Resources/kit/scripts/dkuinstall/migrate_dss.py", line 98, in migrate [Install] reraise(*sys.exc_info()) File "/Applications/DataScienceStudio.app/Contents/Resources/kit/scripts/dkuinstall/migrate_dss.py", line 67, in migrate migration.execute(dip_home, node_type) File "/Applications/DataScienceStudio.app/Contents/Resources/kit/scripts/dkuinstall/migration_base.py", line 136, in execute [Install] wrapper.execute(diphome) File "/Applications/DataScienceStudio.app/Contents/Resources/kit/scripts/dkuinstall/migration_base.py", line 79, in execute migration.execute(ppaths) File "/Applications/DataScienceStudio.app/Contents/Resources/kit/scripts/dkuinstall/migration_json.py", line 132, in execute [Install] migrate_json_file(self, fp) File "/Applications/DataScienceStudio.app/Contents/Resources/kit/scripts/dkuinstall/migration_json.py", line 95, in migrate_json_file new_obj = transformer.transform(cobj, fp) File "/Applications/DataScienceStudio.app/Contents/Resources/kit/scripts/dkuinstall/steps.py", line 4940, in transform [Install] return migrate_ml_spark_params(modeling) File "/Applications/DataScienceStudio.app/Contents/Resources/kit/scripts/dkuinstall/steps.py", line 4907, in migrate_ml_spark_params [Install] spark_params[prop] = obj.get(prop, None) AttributeError: 'NoneType' object has no attribute 'get' [install] EOF reached [install] EOF reached DSS installer terminated with status 1 DSS installer failed , fatal error
Any idea how to fix this?
Thank you!
Best Answer
-
Hi @PGuti
,Thank you for the information!
Can you please check if the params.json file under "/Users/pierregutierrez/Library/DataScienceStudio/dss_home/config/projects/VELIB/analysis/W9NhaPCz/ml/7iRWiMaj/" is null/empty. If so, can you please delete the file or move it outside of the DSS data directory and try again with the migration?
If the issue persists after that, please attach the full log to the chat you opened, so that we can review and investigate further.
Thank you!
Regards,
Huixian Pan
Answers
-
The complete log would show all the project names on a public page which I would prefer avoiding.
The exact line before was:
"""
[Install] Migrating /Users/pierregutierrez/Library/DataScienceStudio/dss_home/config/projects/VELIB/analysis/W9NhaPCz/ml/7iRWiMaj/params.json
"""
The last step written was:
"""
20 -> 21
- Migrates spark params in saved models into a common object
(Wrapped)
"""
I had a bunch of warning messages:
[Install] ** WARNING: Unhandled algorithm: KMEANS[Install] ** WARNING: Unhandled algorithm: EXTRA_TREES
[Install] ** WARNING: Unhandled algorithm: NEURAL_NETWORK
[Install] ** WARNING: Unhandled algorithm: PY_TWO_STEP
which I'm assume are models which changed and custom models.
The beginning of the log seems to explicit what is going to be done and is:
"""
Starting Data Science Studio.app
dssHome:/Users/pierregutierrez/Library/DataScienceStudio/dss_home dssBin:/Users/pierregutierrez/Library/DataScienceStudio/dss_home/bin/dss
Starting Java check
/Applications/DataScienceStudio.app/Contents/Resources/kit/scripts/_find-java.sh
[Install] [+] Using Java at /Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java : java version "1.8.0_121"[Install] /Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java
[Java check] EOF reached
[Java check] EOF reached
Java check terminated with status 0
dss-version exists, doing stop-upgrade-start
Running DSS task for 'stop'
[DSS] dss: DSS supervisor is not running[dss stop] EOF reached
[dss stop] EOF reached
[Install] *********************************************
* Dataiku DSS installer *
*********************************************[Install] [+] /Users/pierregutierrez/Library/DataScienceStudio/dss_home already exists and is a DSS installation
[Install] [+] Saving installation log to /Users/pierregutierrez/Library/DataScienceStudio/dss_home/run/install.log
[Install] [+] Using Java at /Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java : java version "1.8.0_121"
[Install] [+] Checking required dependencies
[Install] + Detected OS distribution : osx 11.6
[Install] + Checking required packages...
[Install] [+] Migrating Python environment
[Install] [+] Precompiling Dataiku Python code
[Install] [+] Precompiling Jupyter Python code
[Install] [+] Precompiling third-party Python 2.7 code
[Install] [+] Migrating data directory
[Install] Select how to migrate from 17 from avail: [8 -> 9, 9 -> 10, 10 -> 11, 11 -> 12, 12 -> 13, 13 -> 14, 14 -> 15, 15 -> 16, 16 -> 17, 17 -> 18, 18 -> 19, 19 -> 20, 20 -> 21, 21 -> 22, 22 -> 23, 23 -> 24, 24 -> 6000, 6000 -> 6020, 6020 -> 6030, 6030 -> 7000, 7000 -> 7020, 7020 -> 8000, 8000 -> 8020, 8999 -> 9000]
Executing the following migrations
17 -> 18
- Clean up the partitioning of SQL datasets
- Upgrade the y/n resize-to-fit setting of dashboard images to new multi-option setting
- Cleanup trained models parameters
- Nest feature generation parameters
- Drop caches
- Drop persistent notifications database
18 -> 19
19 -> 20
- Add params to ML recipes
- Transform comments insight to discussions insights
- Transform dashboard by changing comments insight to discussions insights
- Drop caches
- Drop persistent notifications database
[Install]
20 -> 21
- Migrates spark params in saved models into a common object
- Migrates spark params in analyses' models into a common object
- Migrates spark params in analyses' ml tasks into a common object
- Regroup spark params in recipes
- Drop caches
- Drop persistent notifications database
21 -> 22
- Give NPS Survey Settings to each user
22 -> 23
-
[Install] Setup metadata for code envs in Jupyter kernels' specs
- Upgrade structure of dss_schema_info in H2 databases
- Make CLASS_WEIGHT the default weighting strategy for classification saved models
- Sets a default Git URL whitelist for clones/pulls
- Use smartName in refresh charts steps
- Move ml task param skipExpensiveReports in modeling
- Drop caches
- Drop persistent notifications database
23 -> 24
- Migrating article attachments to use smart IDs
24 -> 6000
- Migrating project path to project folder
- Change project home behavior in user-settings.json
- Migrating article IDs to numeric IDs instead of names
- Migrating homepage articles to ids
[Install][Install] - Migrating dashboard articles to ids
- Upgrade Wiki timeline with numeric article IDs
- Change credentials mode in AWS connections
- Upgrade pre-push hook in general settings
- Upgrade pre-push hook in API deployer
- Migrating visual ML execution params
- Migrating listing of code-envs used in visual deep-learning
- Upgrade evaluation recipe metrics and ouputs configurations
6000 -> 6020
- Migrating article IDs to numeric IDs in dashboards
6020 -> 6030
- Migrating Microsoft Teams scenarios reporters
6030 -> 7000
- Renaming connectionCredentials to credentials for each user
- Convert expositions in infras of API deployer
- Convert expositions in deployments of API deployer
- Removing existing Hipchat Channels
- Removing existing Hipchat Reporters
- Removing existing Hipchat Integrations
- Migrating algorithms params structure
- Migrating posttrain computation params
- Migrating SAML SP configuration
7000 -> 7020
- Migrate scoring recipe explanation params
7020 -> 8000
- Initialize audit settings
- Initialize audit settings (API node)
- Migrating SVM algorithms params structure
- Migrating structure of 'proportion of features to sample'
- Migrating prediction algorithms params structure to add range
- Migrating grid length for non-searchable algos
- Migrate code env selection
8000 -> 8020
- Migrating tree visualisations to take into account difference between class and sample weights
- Rename 'hashSVDhashSize' field to 'hashSize' for text features
- Migrating structure of global tags categories
8999 -> 9000
- Migrate categorical hashing method
- Update preparation script FilterOnDateRange steps
- Update preparation script FlagOnDateRange steps
- Migrating of the jupyter storage path to project
[Install]
- Add parameters to evaluation recipes
- Migrate API service params to use type parameter
- Migrating API Deployer settings to be Deployer settings
- Migrating time triggers
- Migrate max depth for tree based models"""
-
Hello!
So when looking at this file, I saw it was only containing "null". So I deleted it and restarted.
I had the problem for several other files. So I deleted all of them. If I had to guess, I would say it's related to the custom models I had (the big WARNINGS on the log).
In case someone has the same issue as I did, here is the python code to generate all file names you want to remove:
import os import json def load_json_file(filename): with open(filename) as f: data = json.load(f) return data mainpath = "YOURPATHHERE" projects = os.listdir(mainpath) for p in projects: project_path = os.path.join(mainpath,p, "analysis") if os.path.isdir(project_path): analyses = os.listdir(project_path) for analysis in analyses: ml_path = os.path.join(project_path, analysis, "ml") if os.path.isdir(ml_path): models = os.listdir(ml_path) for m in models: final_path = os.path.join(ml_path,m, "params.json") if os.path.isfile(final_path): mydata = load_json_file(final_path) if mydata is None: print("rm " + final_path)
The studio is now working!
Thanks for the help!
-
This is great! Thank you @PGuti
for the updates and thank you for sharing the additional information!Regards,
Huixian Pan