Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I am trying to connect a Python recipe with multiple inputs among which couple of them has to be partitioned so that I can do some transformation on a part of data and then write it to the output dataset.
Things I need to do:
- Partition input dataset on a dimension dynamically. (The list of partition values has to be passed dynamically)
- Read partitioned data into python recipe and do transformations which will be applied on each partitions
- Post transformation, write it to the output dataset.
I came through a solution on this community of using scenario to do it. I am not sure how to do that.
Please let me know if any more detail needs to be provided for clarity.
Operating system used: Windows
Hi @Jurre ,
I referred to the Dataiku community discussion. I have gone through the course link which you provided, didn't find anything concrete there. I want to pass the partition lists dynamically.
Thanks for the help
Hey @Sajid_Khan ,
It really helps to be specific: what have you tried yourself ? What community solution are you referring to, how have you tried to implement that solution ?
There are a lot of resources here to help you, for example :
I hope these resources will hand you some clues on where to start and how to achieve your objective. When posting back please be as specific as possible and include what you have tried yourself. This article is very informative on that matter. Happy coding!
Edit : added link to community resource