How to create parametrized flows in Dataiku ?
How to create parametrized flows in Dataiku ? I want to load different datasets based on the input given. I tried creating a plugin from my custom python recipe and defined 3 input variables as per my requirement. Inside the recipe, I am referring to the method available in my project git repo file which I have added under 'Libraries' in Dataiku. But it still gives me the error that it is unable to find that method/module.
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptionsHi @vaishnavi
,For "parametrized" flows, there would be several features in DSS that can help you accomplish this if I understand the use case.
1. App-as-recipe: https://doc.dataiku.com/dss/latest/applications/application-as-recipe.html2. Plugin recipe: https://doc.dataiku.com/dss/latest/plugins/reference/recipes.html
You seem to have taken the plugin approach.
Plugins are designed to be packable and stand-alone and run on any DSS instance. This means that you can't import project libraries to plugins. You should use the plugin directory to add your code in the python-lib directory of the plugin itself instead.
You also include requirements for plugins as part of the code environment requirements.
Kind Regards,