How to get create date of a scenario

[Marlan]

Hello,

Anyone know how to get the create date of a scenario? We are building some reporting to show utilization of DSS and would like to be able to report information like number of scenarios created this quarter vs. last quarter. 

The modification date is available via the Python API get settings method but not creation date.

Creation date is available in the UI so it's stored somewhere.

I looked at the Scenario Runs internal database thinking that maybe that's a way to approximate creation date (find the first run by scenario) but I'm not sure how long that data is maintained over time. So might work but not ideal.

Any suggestions as how to directly get the scenario create date?

Thanks,

Marlan

[AlexT · Accepted solution]

Hi,

Looking at the Creation/Last Modification dates would be the same so it's slightly misleading that this is actually available in the UI. What you are seeing is the Last modification date that is stored in the metadata and that you retrieve via the API.

We don't actually have the original creation date currently. The only way to retrieve this is actually

1) Search local git commits for "Created scenario"  commit message , iterate trough all projects

e.g 

for dir in DATADIR/config/projects/*; do (cd "$dir" && git log -g --grep="Created scenario" --pretty=format:"%h%x09%an%x09%ad%x09%s"); done 2> /dev/null

Which will output something like which will have the timestamp, projectkey and user that created it :

496c599 admin Sat Nov 13 11:33:58 2021 +0000 Created scenario S3PARTITION.MYSCENARIO

2) Audit logs. message like "/api/scenarios/create" 

Using an audit log would be easier you can simply add the audit logs to DSS as a dataset and parse for this message.  Retention of audit logs can vary on your setup but you have several options to ensure long-term retention of audit logs. See:

https://doc.dataiku.com/dss/latest/operations/audit-trail/centralization-and-dispatch.html#eventserv...

Example of scenario creation message :

"severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/scenarios/create","auditTopic":"apicall","projectKey":"S3PARTITION","authSource":"USER_FROM_UI","data":"{\"type\":\"step_based\",\"name\":\"my-scenario\",\"params\":{},\"id\":\"MYSCENARIO\"}","clientIP":"127.0.0.1","time":255,"xForwardedFor":"127.0.0.1","authUser":"admin","originalIP":"127.0.0.1"},"mdc":{"apiCall":"/api/scenarios/create","user":"admin"},"callTime":255,"timestamp":"2021-11-13T11:33:58.246+0000"}

Hope this helps!

[Marlan]

Thanks @AlexT. Good to know the situation on creation time and as well what options there are to nonetheless get this information. I'm inclined to try the first option as we don't have our audit log tracking configured as would be needed yet. I'll have to work with our admin on how to do that as I of course don't have permissions on those directories from a Jupyter Notebook in DSS.

Thanks!

Marlan