Python recipe with Partitioned input and output

Sajid

Hi

Greetings...

I am trying to connect a Python recipe with multiple inputs among which couple of them has to be partitioned so that I can do some transformation on a part of data and then write it to the output dataset.

Things I need to do:

- Partition input dataset on a dimension dynamically. (The list of partition values has to be passed dynamically)

- Read partitioned data into python recipe and do transformations which will be applied on each partitions

- Post transformation, write it to the output dataset.

I came through a solution on this community of using scenario to do it. I am not sure how to do that.

Please let me know if any more detail needs to be provided for clarity.

Kindly Help,

Thanks,

Sajid Khan

---

Operating system used: Windows

Jurre

Hi @Sajid_Khan
,

which community-solution are you referring to ? The Dataiku Academy has some great resources on partitioning by the way, including a course. -Jurre

Sajid

Hi @Jurre
,

I referred to the Dataiku community discussion. I have gone through the course link which you provided, didn't find anything concrete there. I want to pass the partition lists dynamically.

Thanks for the help

- Sajid

Jurre

Hey @Sajid_Khan
,

It really helps to be specific: what have you tried yourself ? What community solution are you referring to, how have you tried to implement that solution ?

There are a lot of resources here to help you, for example :

• The documentation about partitioning
• Knowledge base articles, for example this or this
• Screencasts,
• and the community forum ofcourse.

I hope these resources will hand you some clues on where to start and how to achieve your objective. When posting back please be as specific as possible and include what you have tried yourself. This article is very informative on that matter. Happy coding!

Edit : added link to community resource