Problems with bundles and plugins in Dataiku migration

Highlighted
Povilas
Level 1
Problems with bundles and plugins in Dataiku migration

Hi.



We are migrating to Dataiku 5.0.2. To do that we are exporting all the projects from our current instance and then importing them to new the instance. There are couple of problems/questions we have:




  • Some our projects have bundles. However after export-import I cannot see the bundles anymore. Maybe you can suggest a way to export-import bundles as well?

  • We have some custom plugins created. Is there any way to export-import them? Or should we recreate each plugin?

  • Also we have some R/Python code environments. Is there a way to export-import them?



Thanks!

0 Kudos
22 Replies
Alex_Combessie Dataiker
Dataiker
Re: Problems with bundles and plugins in Dataiku migration
Hi Povilas, In order to migrate an existing Dataiku installation from one instance to another, we advise to migrate the 'data directory'. This way there is no need for manual ad-hoc migration actions for each project, code environment, configurations, etc. This is documented in more details on https://doc.dataiku.com/dss/latest/installation/migrations.html#migrating-the-data-directory. Hope it helps, Alexandre
0 Kudos
Povilas
Level 1
Re: Problems with bundles and plugins in Dataiku migration
Hi Alex. Thank you! Just to make sure, will this solve our bundles problem too?
0 Kudos
Alex_Combessie Dataiker
Dataiker
Re: Problems with bundles and plugins in Dataiku migration
Hi, this procedure is strictly for migrating projects in a DSS Design node, not Automation node. Is that what you want to achieve? Or do you want to do that from a Design node to an Automation node?
0 Kudos
Povilas
Level 1
Re: Problems with bundles and plugins in Dataiku migration
No, we are migrating from design node to other design node. Just this new node will be on Dataiku 5.0
0 Kudos
Alex_Combessie Dataiker
Dataiker
Re: Problems with bundles and plugins in Dataiku migration
Migrating the data directory can only work with the same Dataiku version. We recommend first to migrate the data directory to another instance with the same version. Then to perform the DSS version upgrade. Hope that clarifies the matter.
0 Kudos
Povilas
Level 1
Re: Problems with bundles and plugins in Dataiku migration
Well, we already have upgraded instance.

However, let's back to bundles. If copy of "data directory" would solve bundles issue (let's say from one instance to the other, both with same DSS version) then those bundles should be stored somewhere in "data directory". Am I correct? If yes, where exactly it is placed?
0 Kudos
Alex_Combessie Dataiker
Dataiker
Re: Problems with bundles and plugins in Dataiku migration
Hi Povilas, bundles is a term only referred to for automation nodes. Are you still referring to a situation with design nodes only? So to clarify you have already upgraded the origin design node to the same version as the destination design node?
0 Kudos
Povilas
Level 1
Re: Problems with bundles and plugins in Dataiku migration
Hi. Yes, I am talking only about design nodes. I don't know how bundles are related with automation node but we used bundles for archiving our projects in design node. For example, we have a project example_project and we want to signicantly upgrade/renew this project. However we don't want to lose our old project or to create "duplicates" like project example_project_v2. So we were creating new bundle for a project everytime we want to make this big upgrade/renew. This was like the way to archive projects. And we had a possibility to restore a project any time to any older version (or bundle).

So basically now we have that history in our design node and we want to be able to tranform that to other instance. Is that possible? I believe it should be, those bundles should be stored somewhere.
0 Kudos
Povilas
Level 1
Re: Problems with bundles and plugins in Dataiku migration
Maybe I already found it. There is a directory '/data/DATA_DIR/bundle_activation_backups/'. I see the bundles here. Maybe it is enough to simply copy this one to new instance?
0 Kudos