Get a metrics value sent via email

Options
Niyazi
Niyazi Registered Posts: 12 ✭✭✭✭

Hi,

I'm trying to use email Reporter to send me values of certain metrics (as an array) when Scenario fails. I've tried using this reference, but it didn't work.

What I tried:

1) Added a step "Define scenario variables" or "Set project variables" and tried both raw and DSS formula to add my variable based on the reference guide above, but it always throws error as follows:

Screenshot 2022-04-01 at 09-58-41 (2) Dataiku Dataiku.png

2) Tried creating a custom variable within the Reporter, but it didn't work either. Maybe I'm misunderstanding the structure of JSON, but error is not very descriptive.

import json# compute your additional variables from the list of report items# and return them as a dictionary.def get_variables(items_json, scenario_run_json, step_run_output_json):step = json.loads(step_run_output_json)if step['MONITORING.sa_connection_tests_NP']['computed'][9]['value']:ds = step['MONITORING.sa_connection_tests_NP']['computed'][9]['value']else:ds = Nonereturn {'failed_ds': ds}

Please advise on how to get it done.


Operating system used: CentOS

Answers

  • dgraham
    dgraham Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 16 Dataiker
    Options

    Hi @zloe
    ,

    The error is caused by an invalid JSON string being entered into the variables field. The value entered must be a valid JSON.

    Additionally, it should be noted that "step_run_output_json" parameter of the custom variables code reporter contains only the output of scenario steps which produce an output, such as the "Execute SQL" scenario step.

    We could add a "Custom Python" scenario step (if supported by your DSS license), and within the Python step we could write code that defines one or more project variables, which contains all or specific metric values of a given dataset in your DSS project. The following example code demonstrates this approach:

    Python script in the "Custom Python" scenario step:

    import dataikuclient = dataiku.api_client()projects = client.get_project(dataiku.default_project_key())dataset = projects.get_dataset("DATASET_NAME") # example to be changedproject_variables = projects.get_variables()# Get all metric values of the datasetmetrics = dataset.get_last_metric_values()all_metrics = metrics.raw['metrics']# Get the history of a specific metric of the datasetrecordCount = dataset.get_metric_history('check:CHECK:Record count')project_variables['standard']['allMetrics'] = all_metrics # update variable valueproject_variables['standard']['recordCount'] = recordCount # update variable valueprojects.set_variables(project_variables) # set project variables

    We could then use these variables as we would normally in the Scenario Reporter.

    2022-04-08_10-35-31.png

Setup Info
    Tags
      Help me…