This website uses cookies. By browsing this website, you consent to the use of cookies. Learn more.
OK
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Join us on July 16th as we explore real-world Reinforcement Learning Learn more

Mont/Day Patitioning

pbazin
Level 1
‎01-29-2020 09:30 AM
Mont/Day Patitioning

Hi,

I have a hierarchy of files in a folders organised by month and day (no year).
Therefore my Partitioning is defined as "%M/%D/.*" but seems not to be correct because the list of aprtitions function returns an error.

Detected 0 partitions

  Found 57941 unmatched files:

  • /01/01/00939537-c3e2-4f9f-8e09-6d0bfdde4ea8_20:42:18.121400/Report_PredictOnRequest.json
  • /01/01/00b9499a-200c-4c0c-91e5-5823ac4f5131_08:20:30.020730/Report_PredictOnRequest.json
  • /01/01/00be9433-5f4b-4a8b-83d0-54f97b74598e_14:22:20.242891/Report_PredictOnRequest.json
  • ...
  • /12/31/303b8500-00bb-46f4-a0d1-5749286ee407_13:21:34.089887/Report_PredictOnRequest.json

What should I do ?

0 Kudos
Reply
2 Replies
Clément_Stenac
Dataiker
Dataiker
‎01-29-2020 09:38 AM

Hi,

Dataiku does not support incomplete time hierarchies (month-day without year).

You will need to manually define two non-time-based partitioning dimensions called "month" and "day", and work manually with them. You will not be able to use time-range management features of DSS.

0 Kudos
Reply
pbazin
Level 1
Author
In response to Clément_Stenac
‎01-29-2020 10:49 AM

I've got my solution.

I could have added a "year" folder on top of my "month/day" hierachy but I'm not at liberty to do it.
What I did is to rename my "month/day" folders as "year-month/day" folders.

Therefore the partitioning definition becomes  "%Y-%M/%D/.*" which works well for DSS.

0 Kudos
Reply