Load different dataset based on parameter from same flow

raviagrawal
raviagrawal Partner, Registered Posts: 18 Partner
Hi,

I am working for a project where I have to create Test & Control data set based of different date parameter but logic to create the dataset is same for both flow. I want to create only one flow so that I do not need to code twice but load it in two different dataset as output. Could you please help me how to create such a flow ?

Answers

  • Alan_Fusté
    Alan_Fusté Partner, Registered Posts: 43 Partner

    Hello raviagrawal,

    The best option for you is to create a Plugin (a Python Custom Recipe). Here https://academy.dataiku.com/latest/tutorial/plugins/recipe.html you can find a tutorial about that.

    As it is explained there, if you have got a Python Code Recipe created, you can convert it to a Plugin by clicking "Convert to Plugin Recipe" (in Advanced tab) and then customize your recipe.json file with custom parameters :)

    Note about parameters:

    The field "params" holds a list of all the params for wich the user will be prompted for values in the Settings tab of the recipe.

    The available parameter types are: STRING, INT, DOUBLE, BOOLEAN, PASSWORD, SELECT, MAP, COLUMN, COLUMNS, TEXTAREA

  • cperdigou
    cperdigou Alpha Tester, Dataiker Alumni Posts: 115 ✭✭✭✭✭✭✭
    You can run your flow on the complete dataset, then split the output dataset into Test and Control results with a split recipe.
  • vaishnavi
    vaishnavi Registered Posts: 40 ✭✭✭✭

    Hi @Alan_Fusté

    I need to implement the similar requirement which @raviagrawal
    had mentioned above.

    Based on the date range I want to fetch the records from a DB tbl and give it as recipe output. I have created a plugin from my python code recipe. I have defined 3 input params. Provided the values of those 3 params in recipe variables under settings which prompted while running the plugin.

    I want to access few files from my project git repo, which is not happening.

    I had also tried adding the git repo link in the plugin as well. But even that dint work. After fetching the git repo inside the plugin, I tried switching to the branch which has updated code. It has broken my plugin and all the settings and code recipe got deleted. When I tried to do something similar to pull the latest changes from git repo added inside plugin, I wasn't able to see my plugin (It got deleted somehow). I later tried to upload the zip file of my plugin, upon clicking the "convert to dev plugin" I am getting an error saying the plugin with same name already exists in dataiku filesystem. But I am not able to see that plugin.

Setup Info
    Tags
      Help me…