Parse Date in-place does not update column schema to Datetime with TZ
Hi,
While following the official Dataiku training dataset and steps, I parsed a date column in-place using “Parse date” with Output type set to “Datetime with tz”. The column visually appeared parsed, but its schema remained string.
This caused a Window recipe to fail, stating the column is string.
Parsing into a new column correctly produces a datetime with tz, which suggests the output type is ignored for in-place parsing. This is not documented and is misleading in the UI.
Is this expected behavior or a bug?
Thanks & Best Regards
Sreejith
Operating system used: Windows
Operating system used: Windows
Answers
-
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,643 NeuronUnfortunately I have seen the same behavior from time to time. I’d agree with you that this is a suboptimal behavior. That said it is fairly easy to fix by changing the storage type and re-saving the recipe. If you can consistently reproduce the error using the provided training instructions and have time. I’d invite you to submit a support ticket letting the Dataiku Team know what you have found.
Other than this, hoping you are having a great day.
—Tom
-
Sreejith Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer Posts: 16 ✭✭✭✭
Hi Tom,
Thank you for the response and for confirming that you’ve seen this behavior as well.
Just to align with your point about it being easy to fix by changing the storage type and re-saving the recipe — in the case where the date is parsed in-place on the same column, this is exactly what I had to do: manually update the column type from the column header menu inside the Prepare recipe, followed by re-saving.
When parsing into a new column, the type is set correctly without any additional action, but for in-place parsing the manual update via the column header appears to be required.
I’ll try to reproduce this consistently using the training dataset and steps, and if I can do so reliably, I’ll submit a support ticket with the details.
Thanks again for the guidance.
Best regards,
Sreejith -
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,643 Neuron
@Sreejith , Even if you can super consistently re-produce this issue. I'd still invite you to submit a support ticket. Reference this post here in the community. I'd also invite others who see this post and have a similar problem to like @Sreejith so that the team at Dataiku know that this has been seen by others.