How to search what recipes the variables were used?

Solved!
Tao_Z999
Level 2
How to search what recipes the variables were used?

My teams defined the variables, but after many versions, I don't know where the variables were used then I can edit or delete, how to search what recipes that the variables were used?


Operating system used: Windows

0 Kudos
1 Solution
Turribeach

Hi, there are no built-in features for this but you can search inside recipe code using the Python API:

 

import dataiku

client_handle = dataiku.api_client()
variables_to_search = ['var1', 'var2']
project_handle = client_handle.get_project('some project key')

python_recipes = [i for i in project_handle.list_recipes() if i['type'] in ['python']]

for python_recipe in python_recipes:
    recipe_name = python_recipe['name']
    recipe_handle = project_handle.get_recipe(recipe_name)
    recipe_script = recipe_handle.get_settings().get_payload().lower()
    if recipe_script:
        for var in variables_to_search:
            if var.lower() in recipe_script:
                print(f'Found variable {var} in recipe {recipe_name}')

 

A less advanced and more basic way will be to export the project, download the export, unzip it and then use Notepad++ Find in Files functionality to search for variables in all the project data files.

 

 

 

View solution in original post

4 Replies
Turribeach

Hi, there are no built-in features for this but you can search inside recipe code using the Python API:

 

import dataiku

client_handle = dataiku.api_client()
variables_to_search = ['var1', 'var2']
project_handle = client_handle.get_project('some project key')

python_recipes = [i for i in project_handle.list_recipes() if i['type'] in ['python']]

for python_recipe in python_recipes:
    recipe_name = python_recipe['name']
    recipe_handle = project_handle.get_recipe(recipe_name)
    recipe_script = recipe_handle.get_settings().get_payload().lower()
    if recipe_script:
        for var in variables_to_search:
            if var.lower() in recipe_script:
                print(f'Found variable {var} in recipe {recipe_name}')

 

A less advanced and more basic way will be to export the project, download the export, unzip it and then use Notepad++ Find in Files functionality to search for variables in all the project data files.

 

 

 

Tao_Z999
Level 2
Author

This works perfectly. I created a python receipt and run this in a jupyter notebook, I also deleted the "python" type filter to scan all the queries like SQL.

0 Kudos
Turribeach

Great, mark my answer as accepted. ๐Ÿ˜‰

0 Kudos
shashank
Dataiker

The non-programmatic way is to create tags for each variable and while developing, tag all those recipes then in the Flow you can set the view to display recipes by each Tag.

0 Kudos