Managed Folder partitions

aydan_kpn

Hi,

I was wondering if it is possible to set multiple time dimensions as partitioning variables in a managed folder?

I am storing pickle files in an s3 managed folder with a folder in folder structure, where the outer folder layer refers to "evaluation_date" and the inner layer refers to "decision_date".

When interacting with this folder I would like to be able to retrieve the appropriate pickle files from it by either filtering on the evaluation date or on decision date. However, when I want to configure the partitions of the folder I seem to only be able to add one time dimension (the outer, "evaluation_date").

Is the a solution you can propose to this issue?

Thank you!

Alexandru

Hi,

Partitions currently only work on a single time dimension. You can submit an idea here

Doing your filtering with python code should work in your case. You can find an example here;

https://community.dataiku.com/t5/Using-Dataiku/Listing-and-Reading-all-the-files-in-a-Managed-Folder/m-p/8140