How to search what recipes the variables were used?
My teams defined the variables, but after many versions, I don't know where the variables were used then I can edit or delete, how to search what recipes that the variables were used?
Operating system used: Windows
Best Answer
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,113 Neuron
Hi, there are no built-in features for this but you can search inside recipe code using the Python API:
import dataiku client_handle = dataiku.api_client() variables_to_search = ['var1', 'var2'] project_handle = client_handle.get_project('some project key') python_recipes = [i for i in project_handle.list_recipes() if i['type'] in ['python']] for python_recipe in python_recipes: recipe_name = python_recipe['name'] recipe_handle = project_handle.get_recipe(recipe_name) recipe_script = recipe_handle.get_settings().get_payload().lower() if recipe_script: for var in variables_to_search: if var.lower() in recipe_script: print(f'Found variable {var} in recipe {recipe_name}')
A less advanced and more basic way will be to export the project, download the export, unzip it and then use Notepad++ Find in Files functionality to search for variables in all the project data files.
Answers
-
This works perfectly. I created a python receipt and run this in a jupyter notebook, I also deleted the "python" type filter to scan all the queries like SQL.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,113 Neuron
Great, mark my answer as accepted.
-
Shashank Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 28 Dataiker
The non-programmatic way is to create tags for each variable and while developing, tag all those recipes then in the Flow you can set the view to display recipes by each Tag.