Tracing Variables

[nharbour]

Is there a feature to trace the use of variables in Dataiku DSS? It would be very useful to be able to automate a search through Projects to identify where it's been used, so we can evaluate the impact of any change to that variable.

[Turribeach · Accepted solution]

Hi, there are no built-in features for this but you can search inside recipe code using the Python API: 

import dataiku

client_handle = dataiku.api_client()
variables_to_search = ['var1', 'var2']
project_handle = client_handle.get_project('some project key')

python_recipes = [i for i in project_handle.list_recipes() if i['type'] in ['python']]

for python_recipe in python_recipes:
    recipe_name = python_recipe['name']
    recipe_handle = project_handle.get_recipe(recipe_name)
    recipe_script = recipe_handle.get_settings().get_payload().lower()
    if recipe_script:
        for var in variables_to_search:
            if var.lower() in recipe_script:
                print(f'Found variable {var} in recipe {recipe_name}')

This being a text search will likely lead to false positives depending on how unique your variables names are. The same code can be used to loop through all projects in your DSS instance but this may take a while to run depending on your instance size.

A less advanced and more basic way will be to export the project, download the export, unzip it and then use Notepad++ Find in Files functionality to search for variables in all the project data files.

[nharbour]

Thank you for your quick response. That's very useful.