Load different dataset based on parameter from same flow

[raviagrawal]

Hi,

I am working for a project where I have to create Test & Control data set based of different date parameter but logic to create the dataset is same for both flow. I want to create only one flow so that I do not need to code twice but load it in two different dataset as output. Could you please help me how to create such a flow ?

[Alan_Fusté]

Hello raviagrawal,

The best option for you is to create a Plugin (a Python Custom Recipe). Here https://academy.dataiku.com/latest/tutorial/plugins/recipe.html you can find a tutorial about that.

As it is explained there, if you have got a Python Code Recipe created, you can convert it to a Plugin by clicking "Convert to Plugin Recipe" (in Advanced tab) and then customize your recipe.json file with custom parameters 🙂

Note about parameters:

The field "params" holds a list of all the params for wich the user will be prompted for values in the Settings tab of the recipe.

The available parameter types are: STRING, INT, DOUBLE, BOOLEAN, PASSWORD, SELECT, MAP, COLUMN, COLUMNS, TEXTAREA

[vaishnavi]

Hi @Alan_Fusté 

I need to implement the similar requirement which @raviagrawal had mentioned above.

Based on the date range I want to fetch the records from a DB tbl and give it as recipe output. I have created a plugin from my python code recipe. I have defined 3 input params. Provided the values of those 3 params in recipe variables under settings which prompted while running the plugin. 

I want to access few files from my project git repo, which is not happening.

I had also tried adding the git repo link in the plugin as well. But even that dint work. After fetching the git repo inside the plugin, I tried switching to the branch which has updated code. It has broken my plugin and all the settings and code recipe got deleted. When I tried to do something similar to pull the latest changes from git repo added inside plugin, I wasn't able to see my plugin (It got deleted somehow). I later tried to upload the zip file of my plugin, upon clicking the "convert to dev plugin" I am getting an error saying the plugin with same name already exists in dataiku filesystem. But I am not able to see that plugin.

[cperdigou]

You can run your flow on the complete dataset, then split the output dataset into Test and Control results with a split recipe.