How to get create date of a scenario

Marlan
Marlan Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Dataiku Frontrunner Awards 2021 Participant, Neuron 2023 Posts: 320 Neuron

Hello,

Anyone know how to get the create date of a scenario? We are building some reporting to show utilization of DSS and would like to be able to report information like number of scenarios created this quarter vs. last quarter.

The modification date is available via the Python API get settings method but not creation date.

Creation date is available in the UI so it's stored somewhere.

I looked at the Scenario Runs internal database thinking that maybe that's a way to approximate creation date (find the first run by scenario) but I'm not sure how long that data is maintained over time. So might work but not ideal.

Any suggestions as how to directly get the scenario create date?

Thanks,

Marlan

Tagged:

Best Answer

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,226 Dataiker
    edited July 17 Answer ✓

    Hi,

    Looking at the Creation/Last Modification dates would be the same so it's slightly misleading that this is actually available in the UI. What you are seeing is the Last modification date that is stored in the metadata and that you retrieve via the API.

    We don't actually have the original creation date currently. The only way to retrieve this is actually

    1) Search local git commits for "Created scenario" commit message , iterate trough all projects

    e.g

    for dir in DATADIR/config/projects/*; do (cd "$dir" && git log -g --grep="Created scenario" --pretty=format:"%h%x09%an%x09%ad%x09%s"); done 2> /dev/null

    Which will output something like which will have the timestamp, projectkey and user that created it :

    496c599 admin Sat Nov 13 11:33:58 2021 +0000 Created scenario S3PARTITION.MYSCENARIO

    2) Audit logs. message like "/api/scenarios/create"

    Using an audit log would be easier you can simply add the audit logs to DSS as a dataset and parse for this message. Retention of audit logs can vary on your setup but you have several options to ensure long-term retention of audit logs. See:

    https://doc.dataiku.com/dss/latest/operations/audit-trail/centralization-and-dispatch.html#eventserver-target

    Example of scenario creation message :

    "severity":"INFO","logger":"dku.audit.apicall","topic":"apicall","message":{"callPath":"/api/scenarios/create","auditTopic":"apicall","projectKey":"S3PARTITION","authSource":"USER_FROM_UI","data":"{\"type\":\"step_based\",\"name\":\"my-scenario\",\"params\":{},\"id\":\"MYSCENARIO\"}","clientIP":"127.0.0.1","time":255,"xForwardedFor":"127.0.0.1","authUser":"admin","originalIP":"127.0.0.1"},"mdc":{"apiCall":"/api/scenarios/create","user":"admin"},"callTime":255,"timestamp":"2021-11-13T11:33:58.246+0000"}

    Hope this helps!

Answers

  • Marlan
    Marlan Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Dataiku Frontrunner Awards 2021 Participant, Neuron 2023 Posts: 320 Neuron

    Thanks @AlexT
    . Good to know the situation on creation time and as well what options there are to nonetheless get this information. I'm inclined to try the first option as we don't have our audit log tracking configured as would be needed yet. I'll have to work with our admin on how to do that as I of course don't have permissions on those directories from a Jupyter Notebook in DSS.

    Thanks!

    Marlan

Setup Info
    Tags
      Help me…