Does Schema propagation work on existing columns in a prepare recipe
Hi All,
I had a question on whether schema propagation works in the prepare recipe ?
For if there is a column in the input dataset which has a datatype of BIGINIT then during the first run the output dataset will also have the datatype as BIGINT
But during a future run if the datatype of the column in the input dataset changes to DOUBLE , then this change doesnot propagate through to the output dataset. The column in the output dataset remains BIGINT and hence the values end up getting dropped.
Hence if anyone has any suggestions or has an idea if this has already been discussed in any other thread then please do guide me in the right direction.
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,111 Neuron
What Dataiku version are you on? There has been some fixes around schema propagation in v12:
-
Hi @Turribeach
Yes i am on V12 . Tried this but somehow it doesnt work for the prepare recipe.
If you believe it works at your end can you please let me know.