Get a metrics value sent via email

Niyazi
Niyazi Registered Posts: 12 ✭✭✭✭
edited July 16 in Using Dataiku

Hi,

I'm trying to use email Reporter to send me values of certain metrics (as an array) when Scenario fails. I've tried using this reference, but it didn't work.

What I tried:

1) Added a step "Define scenario variables" or "Set project variables" and tried both raw and DSS formula to add my variable based on the reference guide above, but it always throws error as follows:

Screenshot 2022-04-01 at 09-58-41 (2) Dataiku Dataiku.png

2) Tried creating a custom variable within the Reporter, but it didn't work either. Maybe I'm misunderstanding the structure of JSON, but error is not very descriptive.

import json
# compute your additional variables from the list of report items 
# and return them as a dictionary.
def get_variables(items_json, scenario_run_json, step_run_output_json):
    step = json.loads(step_run_output_json)
    if step['MONITORING.sa_connection_tests_NP']['computed'][9]['value']:
        ds = step['MONITORING.sa_connection_tests_NP']['computed'][9]['value']
    else:
        ds = None
    return {'failed_ds': ds}

Please advise on how to get it done.


Operating system used: CentOS

Answers

  • dgraham
    dgraham Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 16 Dataiker
    edited July 17

    Hi @zloe
    ,

    The error is caused by an invalid JSON string being entered into the variables field. The value entered must be a valid JSON.

    Additionally, it should be noted that "step_run_output_json" parameter of the custom variables code reporter contains only the output of scenario steps which produce an output, such as the "Execute SQL" scenario step.

    We could add a "Custom Python" scenario step (if supported by your DSS license), and within the Python step we could write code that defines one or more project variables, which contains all or specific metric values of a given dataset in your DSS project. The following example code demonstrates this approach:

    Python script in the "Custom Python" scenario step:

    import dataiku
    client = dataiku.api_client()
    projects =  client.get_project(dataiku.default_project_key())
    dataset = projects.get_dataset("DATASET_NAME") # example to be changed
    project_variables = projects.get_variables()
    
    # Get all metric values of the dataset
    metrics = dataset.get_last_metric_values()
    all_metrics = metrics.raw['metrics']
    
    # Get the history of a specific metric of the dataset
    recordCount = dataset.get_metric_history('check:CHECK:Record count')
    
    project_variables['standard']['allMetrics'] = all_metrics # update variable value
    project_variables['standard']['recordCount'] = recordCount # update variable value
    projects.set_variables(project_variables) # set project variables

    We could then use these variables as we would normally in the Scenario Reporter.

    2022-04-08_10-35-31.png

Setup Info
    Tags
      Help me…