Announcing the winners & finalists of the Dataiku Frontrunner Awards 2021! Read their inspiring stories

Propogate locked data type through the flow

bcalcutt
Level 1
Propogate locked data type through the flow

I have a manually set data type in my input data set, but it gets auto-converted back every time the data set goes through a recipe. Is there any way to propagate the locked data type through the data flow? Is there any work around besides having to lock the data type at every data set in the flow?

0 Kudos
3 Replies
natejgardner
Level 6

When you change a type, if you open the next recipe downstream and run it from there, you'll be prompted to accept the schema change. You need to do this for each recipe downstream to propagate your schema changes. Another approach is to use the API. You can fetch the schema dictionary from your source dataset and update the type of that column in all the downstream schema dictionaries with a Python script.

I've also often wished for the ability to propagate changes to a schema downstream with a single click. You can suggest that feature as a product idea here.

0 Kudos
natejgardner
Level 6

Actually, does this context menu item do what you're looking for? I only noticed it today.

 

 

Screenshot 2021-07-22 140242.jpg

0 Kudos
bcalcutt
Level 1
Author

I was going to reply with this option, but unfortunately it does not propagate the manually selected data types. I’m not sure why though.

0 Kudos
A banner prompting to get Dataiku DSS