Submit your inspiring success story or innovative use case to the 2022 Dataiku Frontrunner Awards! ENTER YOUR SUBMISSION

How to migrate from one database to another

mpemburn
Level 2
How to migrate from one database to another

Hi,

Please forgive me if this has been asked a million times, but I was unable to find what I was looking for with the searches I tried.

In my last job, our data specialist used DSS to migrate data from a legacy database (in Paradox, I believe) to MySql.  There were a huge number of differences between the two schemas, but he was able to accomplish everything that was needed with this marvelous tool!

I now have a similar task—migrating between an old and new MySql db—but I'm having a hard time figuring out the next step(s).  I have connections to both databases, and I've added the tables from the source database to my project.  At this point, I have no idea how to add the second connection (to the destination db).

Edited to add: I've been able to add datasets from both the source and destination databases but I need some guidance on what kind of recipes will allow me to modify and export data from one schema to the other.  I've included a screen shot showing part of two schemas.

Any help you can offer would be greatly appreciated!

Regards,

Mark Pemburn

0 Kudos
26 Replies
mpemburn
Level 2
Author

Hi @Ignacio_Toledo,

Yes, that did import—thanks!  It still complained about missing plugins, andI was able to install github and confluence, but DSS couldn't find any of the others.

Mark

0 Kudos
Ignacio_Toledo

Hi @mpemburn. Great news! And don't worry about the plugins warnings: since they are not used by this project we are safe that no problems will happen.

OK, I'll keep moving forward now. 

0 Kudos
mpemburn
Level 2
Author

Hi @Ignacio_Toledo,

I though surely that I'd replied, but apparently I was dreaming at the time 😉  So, your latest project file installed with no problem!  What's next?

Regards,

Mark

0 Kudos
Ignacio_Toledo

Hello @mpemburn,

I've been working on the project this weekend, adding some information in its wiki and extending the Flow. I hope to send something by the end of the day, and hopefully we can work together or include more people in the project.

The idea remains the same: is an example of ETL to go from one schema and DB to another. It might not solve your particular case directly, but the idea is to show the available methods in DSS

Cheers

I.

0 Kudos
Ignacio_Toledo

By the way, what I'm trying to do here is to move from the sakila original schema:

dbschema5.png

to a new Star schema that will look like this:

pagila-star.png

 By using this publicly available example dataset (sakila) we can work together on data that is safe, public and common.

Let me know if you think this is a good idea for you. Or if you were looking to work directly with your own data.

0 Kudos
Ignacio_Toledo

Hi @mpemburn,

I hope you are doing well. I just wanted to find out how have you moved forward, and if you think that we should meet again to work in the "example project".

Cheers!

0 Kudos