Monitor users

reneege
reneege Registered Posts: 23 ✭✭✭✭

We have just deployed a couple of DSS projects to our automation node, and given Reader access to many customers who will be viewing the dashboards or running webApps produced by those projects.

We are wondering what is the best way to monitor the usages from those users. Is there a way we can get a report on who has logged on and what activities during last week/month without digging into log files?

We are on DSS 6.0.1. Any information would be appreciated!

Best Answer

  • Clément_Stenac
    Clément_Stenac Dataiker, Dataiku DSS Core Designer, Registered Posts: 753 Dataiker
    edited July 17 Answer ✓

    Hi,

    We do not have a reference of the audit log.

    However, as a quick start, each JSON record in the audit log looks like:

    {"severity":"INFO","logger":"dku.audit.generic","message":{"callPath":"/api/admin/logs/get-content","msgType":"admin-log-get","authSource":"USER_FROM_UI","clientIP":"127.0.0.1","name":"audit/audit.log","authUser":"admin","originalIP":"90.3.190.212"},"mdc":{"apiCall":"/api/admin/logs/get-content","user":"admin"},"callTime":2,"timestamp":"2020-04-19T04:39:40.340-0400"}

    • You want to focus the ones where "logger" is "dku.audit.generic".
    • Generally speaking, you can ignore "severity", "callTime", "mdc" and "message.callPath" which are fairly internal details

    • Then, the most important pieces of information are:timestamp, message.authUser and message.msgType

    These three items make up the "when", "who" and "what" of the activity. Then, depending on "msgType", you will have additional details within "message".

    The granularity of audit event is very fine and each page view will trigger multiple audit events corresponding to the underlying API calls. You'll thus indeed want to focus on some specific audit events.

    Most events will at least contain a "projectKey" element in message, and will contain information about the current object. For example, all dataset-related events will contain "datasetName" and so on.

    Here are some "interesting" audit events "msgType"

    • application-open: DSS was open in a browser tab
    • login/logout: self-explanatory
    • dataset-read-data-sample: A dataset's Explore was open
    • flow-job-start / flow-job-done: A job was started/completed
    • flow-object-build-start / flow-object-build-failed / flow-object-build-success: Within a job, a dataset was built
    • scenario-run: A scenario was run manually
    • scenario-fire-trigger: A scenario was run automatically

Answers

  • ATsao
    ATsao Dataiker Alumni, Registered Posts: 139 ✭✭✭✭✭✭✭✭

    Hi there,

    This information will indeed be contained in the audit logs, which can be found in the "run" folder of your DSS data directory. However, instead of parsing them manually, you can always import these files as datasets in DSS and use DSS to perform the analysis you need! A bit meta but I hope that this helps or at least gets you pointed in the right direction!

    Best,

    Andrew

  • reneege
    reneege Registered Posts: 23 ✭✭✭✭

    Thank you for your info! My understanding that this tool is for monitoring DSS services and system resource usages. Our interest is user logins and activities in a summarized report.

  • Marlan
    Marlan Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Dataiku Frontrunner Awards 2021 Participant, Neuron 2023 Posts: 320 Neuron

    Hi Andrew,

    We have in fact done this (analyze the most recent audit log by connecting to it in DSS). However, making sense of the audit log has been a challenge. Is there any reference to how to interpret data in the audit logs? We are interested in something similar - logins and key activities (building datasets and running scenarios would be a start). The basic question is who is using DSS, how much are using it, and for what.

    Thanks,

    Marlan

  • reneege
    reneege Registered Posts: 23 ✭✭✭✭

    Thank you much for this info, which helps us to understand the audit log details better! Hope Dataiku will make an enhancement in near future to add a summary dashboard/report for user activities per projects under 'Administration -> Maintenance".

  • Marlan
    Marlan Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Dataiku Frontrunner Awards 2021 Participant, Neuron 2023 Posts: 320 Neuron

    Hi @Clément_Stenac
    ,

    This is super helpful. I had made some progress in understanding the log but your notes will really in putting together some helpful monitoring. Appreciate you taking the time to write these up.

    Thank you!

    Marlan

  • CoreyS
    CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭

    We're happy you find this helpful @Marlan
    !

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,124 Neuron

    I second this request, Dataiku should provide better data on how users are using the app. Audit logs are not really suited for that although they can be used.

Setup Info
    Tags
      Help me…