Survey banner
The Dataiku Community is moving to a new home! Some short term disruption starting next week: LEARN MORE

Load different dataset based on parameter from same flow

Level 3
Load different dataset based on parameter from same flow

I am working for a project where I have to create Test & Control data set based of different date parameter but logic to create the dataset is same for both flow. I want to create only one flow so that I do not need to code twice but load it in two different dataset as output. Could you please help me how to create such a flow ?
3 Replies
Level 3

Hello raviagrawal,

The best option for you is to create a Plugin (a Python Custom Recipe). Here you can find a tutorial about that.

As it is explained there, if you have got a Python Code Recipe created, you can convert it to a Plugin by clicking "Convert to Plugin Recipe" (in Advanced tab) and then customize your recipe.json file with custom parameters 🙂

Note about parameters:

The field "params" holds a list of all the params for wich the user will be prompted for values in the Settings tab of the recipe.


0 Kudos
Level 3

Hi @Alan_Fusté 

I need to implement the similar requirement which @raviagrawal had mentioned above.

Based on the date range I want to fetch the records from a DB tbl and give it as recipe output. I have created a plugin from my python code recipe. I have defined 3 input params. Provided the values of those 3 params in recipe variables under settings which prompted while running the plugin. 

I want to access few files from my project git repo, which is not happening.


I had also tried adding the git repo link in the plugin as well. But even that dint work. After fetching the git repo inside the plugin, I tried switching to the branch which has updated code. It has broken my plugin and all the settings and code recipe got deleted. When I tried to do something similar to pull the latest changes from git repo added inside plugin, I wasn't able to see my plugin (It got deleted somehow). I later tried to upload the zip file of my plugin, upon clicking the "convert to dev plugin" I am getting an error saying the plugin with same name already exists in dataiku filesystem. But I am not able to see that plugin.

0 Kudos
Dataiker Alumni
You can run your flow on the complete dataset, then split the output dataset into Test and Control results with a split recipe.
0 Kudos