Discover this year's submissions to the Dataiku Frontrunner Awards and give kudos to your favorite use cases and success stories!READ MORE

Load different dataset based on parameter from same flow

raviagrawal
Level 3
Level 3
Load different dataset based on parameter from same flow
Hi,

I am working for a project where I have to create Test & Control data set based of different date parameter but logic to create the dataset is same for both flow. I want to create only one flow so that I do not need to code twice but load it in two different dataset as output. Could you please help me how to create such a flow ?
3 Replies
Alan_Fusté
Level 3
Level 3

Hello raviagrawal,



The best option for you is to create a Plugin (a Python Custom Recipe). Here https://academy.dataiku.com/latest/tutorial/plugins/recipe.html you can find a tutorial about that.



As it is explained there, if you have got a Python Code Recipe created, you can convert it to a Plugin by clicking "Convert to Plugin Recipe" (in Advanced tab) and then customize your recipe.json file with custom parameters 🙂



Note about parameters:



The field "params" holds a list of all the params for wich the user will be prompted for values in the Settings tab of the recipe.

The available parameter types are: STRING, INT, DOUBLE, BOOLEAN, PASSWORD, SELECT, MAP, COLUMN, COLUMNS, TEXTAREA

0 Kudos

Hi @Alan_Fusté 

I need to implement the similar requirement which @raviagrawal had mentioned above.

Based on the date range I want to fetch the records from a DB tbl and give it as recipe output. I have created a plugin from my python code recipe. I have defined 3 input params. Provided the values of those 3 params in recipe variables under settings which prompted while running the plugin. 

I want to access few files from my project git repo, which is not happening.

 

I had also tried adding the git repo link in the plugin as well. But even that dint work. After fetching the git repo inside the plugin, I tried switching to the branch which has updated code. It has broken my plugin and all the settings and code recipe got deleted. When I tried to do something similar to pull the latest changes from git repo added inside plugin, I wasn't able to see my plugin (It got deleted somehow). I later tried to upload the zip file of my plugin, upon clicking the "convert to dev plugin" I am getting an error saying the plugin with same name already exists in dataiku filesystem. But I am not able to see that plugin.

0 Kudos
cperdigou
Dataiker Alumni
You can run your flow on the complete dataset, then split the output dataset into Test and Control results with a split recipe.
0 Kudos