New to Dataiku DSS? Try out our NEW Quick Start Programs today and get onboarded on the product in just one hour! Let's go

Accessing metrics in reporters when partitioned datasets are used

dmajard
Level 2
Accessing metrics in reporters when partitioned datasets are used
Hi,

I had a reporter that worked marvels until I started keeping historical records through partitioning. I cannot find the way to dynamically parse the ever changing json. What was once:

${(filter(parseJson(stepOutput_get_counts)["MY_PROJECT.my_table_NP"].computed, x,...

is now:

${(filter(parseJson(stepOutput_get_counts)["MY_PROJECT.my_table_2019-03-12"].computed, x,...

how can I get the CURRENT_DAY partition in the name there?



Cheers,

Dany
0 Kudos
1 Reply
Alex_Combessie
Dataiker
Dataiker
Hello,

For advanced usage of metrics such as partitioning, I would not advise using the ${} magic you were using before partitioning.

Instead, you can add a Python step in your scenario which retrieves the value of your metric for your CURRENT_DAY partition, and store it in a project variable called "myvar". You would then be able to use ${myvar} in your reporter. That way you have the full flexibility and simplicity of the Dataiku public API in your Python step. See https://doc.dataiku.com/dss/latest/python-api/rest-api-client/metrics-and-checks.html for reference. You can easily prototype it first in a Python notebook, and then paste it into the Python scenario step.

Hope it helps,

Alex
0 Kudos