Survey banner
Share your feedback on the Dataiku documentation with this 5 min survey. Thanks! TAKE THE SURVEY

Add hour, minute, and second to shaker increment date processor

Edit: I found out that the formula processor's inc() function can handle these units, so the value of adding this into the increment date processor as well is primarily just to reduce confusion and improve consistency. I'm fairly certain I'd used the inc() function in the formula processor before, but having forgotten about it and seen the increment date processor right there in the processor library, I assumed it was currently the only way. So this idea is mainly just a UX improvement.

 

--

Currently, the increment date processor in shaker recipes can only increment by year, quarter, month, week, and day. But because of leap seconds and daylight savings time, it's common to need to increment by hours, minutes, and seconds as well while preserving calendar accuracy. It would be great if these could be added.

Currently, I have a list of dated events in the future, all in local times. I have another column containing the current time zone for each event, but I need to compute the future time zone for those events due to daylight savings. I have a column containing the end date for a given timezone, and if the date of the event is greater than the end date for the current timezone, I will flip the DST value by subtracting or adding one hour to the converted UTC time based on the nominal timezone. This will allow me to get the true UTC time of any of these events I only have in local times at various locations. The increment date feature would be perfect for this if it supported smaller units. Unfortunately, noninteger values here are rounded down. Python libraries can do the same thing, but these tend to have a lot of overhead for prepare recipes and slow down execution, in addition to adding complexity. Ideally, this could be handled directly by the increment date processor.

3 Comments
Jason
Level 4

Yeah +1 on this, I just ran into this same issue.  I'm making predictions in minutes... and I need to add the prediction to a known date to get the predicted date.  I'd love to have this in a shaker script.

Yeah +1 on this, I just ran into this same issue.  I'm making predictions in minutes... and I need to add the prediction to a known date to get the predicted date.  I'd love to have this in a shaker script.

ktgross15
Dataiker

Thanks for the feedback, this has been added to our backlog and we will keep you updated if anything changes.

Katie

Status changed to: In Backlog

Thanks for the feedback, this has been added to our backlog and we will keep you updated if anything changes.

Katie

MichaelG
Community Manager
Community Manager
 
I hope I helped! Do you Know that if I was Useful to you or Did something Outstanding you can Show your appreciation by giving me a KUDOS?

Looking for more resources to help you use DSS effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as ‘Accepted Solution’ to help others like you!
Status changed to: In the Backlog