Dataset using partition

Ankur5289
Level 3
Dataset using partition

I have a dataset which i need to partition based on timestamp . So there is a column called time stamp and i am using partition to create data set. The partitioning type is Time Range and the range selection is Hour.

This is being created as mongo Db type dataset

i am getting error as below:

An invalid argument has been encountered : Illegal time partitioning value : '2021-05-26T13:37:39.861' with a HOUR partitioning

Has anyone faced similar issue and any leads to solve this - to get the data partitioned based on HOUR time range?

0 Kudos
1 Reply
fchataigner2
Dataiker

Hi

the format of the values expected in the column used as time partitioning source is year-month-day-hour (ie the components of the partitioning value, separated by dashes). 

If the field of the collection you're using doesn't conform to this format, you should create a view to add a derived field with the proper format, and create the DSS dataset on the view, like 

db.createView ("item_view", "item", [{$addFields:{sourceTimestampPartition:{$dateToString:{date:"$sourceTimestamp",format:"%Y-%m-%d-%H"}}}}])

0 Kudos