Ready for Dataiku 10? Try out the Crash Course on new features!GET STARTED

How to use available variables in reporters section/Send to dataset in scenarios ?

Solved!
BlueRabit
Level 2
How to use available variables in reporters section/Send to dataset in scenarios ?

Hi, I would like to create a scenario which allows me to have kind of a history dataset build in outputs flow checks. The idea is to trigger the scenario once every week, build some dataset over the flow, run checks,... and at the end if all went good, add a new row in a dataset with the date and the hour of the execution of the scenario, the output of scenario, and more.

I have noticed that we have some variables available in the reporter, that's good!

Now I would like to use them, but foremost understand their JSON constitution in order to get work autonomously.

 

 

 

0 Kudos
1 Solution
dgraham
Dataiker
Dataiker

Hi @BlueRabit ,

The "Send to dataset" reporter adds a new row to an existing dataset  at the beginning or end of the scenario run. Note, the Dataset (as specified in the "Dataset name" field) must point to a Dataset that is on a writable connection (i.e. "Allow write" must be enabled under "Usage params" of the the connection settings). Additionally, the Timestamp column must be a date column (i.e. storage type of Date) in the dataset. 

As for your question, the JSON specified in the input text area should be a JSON string with one or more key-value pairs that correspond to the <column_name> : <value> for the row which is added to the dataset. For example, if we had a dataset with the following three columns: Timestamp, Scenario, Status

 

example_dataset.png

 

Then we could configure the send to dataset reporter with the following JSON:

example_send_to_dataset.png

{
  "Scenario": "${scenarioName}",
  "Status": "${outcome}"
}
 

 

 

 

View solution in original post

0 Kudos
4 Replies
dgraham
Dataiker
Dataiker

Hi @BlueRabit ,

The "Send to dataset" reporter adds a new row to an existing dataset  at the beginning or end of the scenario run. Note, the Dataset (as specified in the "Dataset name" field) must point to a Dataset that is on a writable connection (i.e. "Allow write" must be enabled under "Usage params" of the the connection settings). Additionally, the Timestamp column must be a date column (i.e. storage type of Date) in the dataset. 

As for your question, the JSON specified in the input text area should be a JSON string with one or more key-value pairs that correspond to the <column_name> : <value> for the row which is added to the dataset. For example, if we had a dataset with the following three columns: Timestamp, Scenario, Status

 

example_dataset.png

 

Then we could configure the send to dataset reporter with the following JSON:

example_send_to_dataset.png

{
  "Scenario": "${scenarioName}",
  "Status": "${outcome}"
}
 

 

 

 

0 Kudos
BlueRabit
Level 2
Author

Thanks a lot for your informations, I figured it out my self in mid time what you told me, but still appreciate !

0 Kudos
BlueRabit
Level 2
Author

Hi again, I solved my problem, thanks to you !
I have an other prob now, Im using a variable has we saw earlier in the column "timestamp" so I can save the date and hour of scenario run.

My probleme is, the hours saved are not corresponding to the actual hours, there is an hour difference between the actual execution time and the time recorded in my dataset.

For example, when a scenario runs at 8AM, I'll have a new row in my dataset with a timestamp registered at 7AM.

Do you know why ? @dgraham 

Thanks for answers

 

0 Kudos
dgraham
Dataiker
Dataiker

Hi @BlueRabit ,

Timestamps are added to the Timestamp column of the dataset following the UTC time standard. Depending on your geographic location, this may explain the 1 hour difference you observe.  

0 Kudos

Labels

?
Labels (2)
A banner prompting to get Dataiku DSS