How to view how many records changes each steps run in each scenerio?

dave

Hello Dataiku Masters,

I would like to see if there are any scenario runs with cutome trigger or time based trigger i would like to see how many data rows changes after each steps or total no. rows after finishing the entire scenario, possible to see?

Ignacio_Toledo

Hi @dave
. Just to be sure, you want to have access to the history of the dataset changes after each scenario run? For example, would the following screenshot fit your need?

If yes, let me know and I can show you how one member of our team did achieve that using the Dataiku capabilities.

Cheers!

dave

Hello@Igancio,

What am really looking is -After each scenario runs how many new records has been overwritten or newly added/appended if its or maybe if we can see by drill down at each job may be under one scenarios so you can have the trend of scenarios to compare the record count? so in nutshell scenario to scenario no. of records added/overwritten etc kind of comparison to understand overall data set behaviour

Hope am able to clear the query.

Ignacio_Toledo

Hi @dave
, yes I think I understand what you are looking for. The screenshot I shared with you is showing one point after each run of the scenario (once a day) for a particular dataset (missing_qa0_tags in the example). That is the visual representation, but the data being plotted can be exported as a dataset, like the file attached which is an excel export of this dataset of metrics.

Each scenario run is identified by the 'timeComputed' column (the time at which the run was made), and for each run you have a set of metrics. For example, 'records:COUNT_RECORDS', which tells you how many records the dataset has after each run, and in this way you could create a trend of home many records were added or removed each day.

I think this information will allow you to get the answer you are looking for. If you think so, I can post a short 'screencast' showing how we are doing it with DSS.

Cheers!

PK36313

Hi @Ignacio_Toledo
,

I am also having similar kind of requirement , I want to tack percentage of increase or decrease in record count every time when the scenario will run .

I am trying to achieve it though metrics and check but not able to do , because in check we can't get the previous record count.

Could you please tell me some way out to do this , it would be great help

Thanks in Advance