You now have until September 15th to submit your use case or success story to the 2022 Dataiku Frontrunner Awards!ENTER YOUR SUBMISSION

Store datetime value as "local" time

When reading data sources, it is possible to check a box that tells Dataiku to understand the date as "local" time.  "Local" being local to the server time zone running the job.  But if I'm going to use these dates to make predictions (in my case, I predict a time delta, then add that delta to the original timestamp, which provides me with the predicted date and time that an event will occur on) then ultimately I need the prediction to be localized in the same manor the original date was.  Presently, I have to manually increment that predicted date by the time zone offset, and because this is occurring in a time zone that shifts for daylight savings, the offset value changes periodically.  It would be a really nice feature to simply be able to designate that the output time is timezoneless (and in doing so, is automatically shifted back from Zulu time to the server's local time zone).

I know I can handle all of this with some prepare recipes and scenario variables, but it would be a lot smoother (and more easily understood by someone who may need to update the project later) to have this handled natively.

1 Comment
Marlan
Neuron
Neuron

I would love to have this option as well. Thanks for suggesting it @Jason.

We are migrating our data storage to Snowflake and find that the Snowflake timestamp without time zone data type is the best option all around for our purposes - except that DSS treats it as a string rather than a date for managed datasets.

It'd be great to have the option to tell DSS to treat TIMESTAMPNTZ types in managed datasets as date column just like one can when defining an external dataset on a Snowflake table. Ideally this could be done at the instance level as we'd just turn it on for our use company wide. 

Note that there is a workaround which is to use the dataset overrides functionality. One can specify params.readColsWithUnknownTzAsDates in the path box and true in the value box to have DSS treat the column as a date.  

Thanks,

Marlan