Propogate locked data type through the flow
I have a manually set data type in my input data set, but it gets auto-converted back every time the data set goes through a recipe. Is there any way to propagate the locked data type through the data flow? Is there any work around besides having to lock the data type at every data set in the flow?
Answers
-
When you change a type, if you open the next recipe downstream and run it from there, you'll be prompted to accept the schema change. You need to do this for each recipe downstream to propagate your schema changes. Another approach is to use the API. You can fetch the schema dictionary from your source dataset and update the type of that column in all the downstream schema dictionaries with a Python script.
I've also often wished for the ability to propagate changes to a schema downstream with a single click. You can suggest that feature as a product idea here.
-
Actually, does this context menu item do what you're looking for? I only noticed it today.
-
I was going to reply with this option, but unfortunately it does not propagate the manually selected data types. I’m not sure why though.
