Accessing metrics in reporters when partitioned datasets are used

[dmajard]

Hi,

I had a reporter that worked marvels until I started keeping historical records through partitioning. I cannot find the way to dynamically parse the ever changing json. What was once:

${(filter(parseJson(stepOutput_get_counts)["MY_PROJECT.my_table_NP"].computed, x,...

is now:

${(filter(parseJson(stepOutput_get_counts)["MY_PROJECT.my_table_2019-03-12"].computed, x,...

how can I get the CURRENT_DAY partition in the name there?



Cheers,

Dany

[Alex_Combessie]

Hello,

For advanced usage of metrics such as partitioning, I would not advise using the ${} magic you were using before partitioning.

Instead, you can add a Python step in your scenario which retrieves the value of your metric for your CURRENT_DAY partition, and store it in a project variable called "myvar". You would then be able to use ${myvar} in your reporter. That way you have the full flexibility and simplicity of the Dataiku public API in your Python step. See https://doc.dataiku.com/dss/latest/python-api/rest-api-client/metrics-and-checks.html for reference. You can easily prototype it first in a Python notebook, and then paste it into the Python scenario step.

Hope it helps,

Alex