How to create partition in python?

mi6crazyheart

New to Dataiku. Little context - I've one Python script which fetch the data from one external data source & creates some compressed CSV files with file name having specific pattern. Ex: aaa_file_1.csv.gz, aaa_file_2.csv.gz etc. Now, I need to take this script inside Dataiku & need to do the same thing using dataiku "partition".

From the documentation, what I can able to understand is after creating the those compressed csv files inside a managed folder I need to enable the partition inside that which I don't know how to do using Python. I hope I'm right at here.

Need some guidance. Some sample code will be a great help.

Thanks

MikeG

Hi @mi6crazyheart
,

Thanks for reaching out. I’m looking at this case and I wanted to collect a few more details about your goal before offering a solution:

1. Is your goal to split a single csv into multiple csv files (along the dimension selected for partitioning) within a managed folder within DSS programmatically? Or,
2. Have you already generated the multiple csv files outside of DSS and you want to copy them to a managed folder within DSS programmatically?
3. If you can share it, a copy of your pre-existing python script could be helpful for me to better understand your goal.

Note: in case it’s helpful I’m leaving a link to a free training module on partitioning in DSS ( I found this helpful when I was initially learning about partitioning in DSS): https://academy.dataiku.com/path/advanced-designer/advanced-partitioning