Monitor users

[reneege]

We have just deployed a couple of DSS projects to our automation node, and given Reader access to many customers who will be viewing the dashboards or running webApps produced by those projects.

We are wondering what is the best way to monitor the usages from those users. Is there a way we can get a report on who has logged on and what activities during last week/month without digging into log files?

We are on DSS 6.0.1. Any information would be appreciated!

[ATsao]

Hi there,

This information will indeed be contained in the audit logs, which can be found in the "run" folder of your DSS data directory. However, instead of parsing them manually, you can always import these files as datasets in DSS and use DSS to perform the analysis you need! A bit meta but I hope that this helps or at least gets you pointed in the right direction!

Best,

Andrew

[reneege]

Thank you for your info!  My understanding that this tool is for monitoring DSS services and system resource usages. Our interest is user logins and activities in a summarized report.

[Marlan]

Hi Andrew,

We have in fact done this (analyze the most recent audit log by connecting to it in DSS). However, making sense of the audit log has been a challenge. Is there any reference to how to interpret data in the audit logs? We are interested in something similar - logins and key activities (building datasets and running scenarios would be a start). The basic question is who is using DSS, how much are using it, and for what.

Thanks,

Marlan 

[Clément_Stenac · Accepted solution]

Hi,

We do not have a reference of the audit log.

However, as a quick start, each JSON record in the audit log looks like:

{"severity":"INFO","logger":"dku.audit.generic","message":{"callPath":"/api/admin/logs/get-content","msgType":"admin-log-get","authSource":"USER_FROM_UI","clientIP":"127.0.0.1","name":"audit/audit.log","authUser":"admin","originalIP":"90.3.190.212"},"mdc":{"apiCall":"/api/admin/logs/get-content","user":"admin"},"callTime":2,"timestamp":"2020-04-19T04:39:40.340-0400"}

• You want to focus the ones where "logger" is "dku.audit.generic".

• Generally speaking, you can ignore "severity", "callTime", "mdc" and "message.callPath" which are fairly internal details

• Then, the most important pieces of information are:timestamp, message.authUser and message.msgType

These three items make up the "when", "who" and "what" of the activity. Then, depending on "msgType", you will have additional details within "message".

The granularity of audit event is very fine and each page view will trigger multiple audit events corresponding to the underlying API calls. You'll thus indeed want to focus on some specific audit events.

Most events will at least contain a "projectKey" element in message, and will contain information about the current object. For example, all dataset-related events will contain "datasetName" and so on.

Here are some "interesting" audit events "msgType"

• application-open: DSS was open in a browser tab
• login/logout: self-explanatory
• dataset-read-data-sample: A dataset's Explore was open
• flow-job-start / flow-job-done: A job was started/completed
• flow-object-build-start / flow-object-build-failed / flow-object-build-success: Within a job, a dataset was built
• scenario-run: A scenario was run manually
• scenario-fire-trigger: A scenario was run automatically

[Marlan]

Hi @Clément_Stenac ,

This is super helpful. I had made some progress in understanding the log but your notes will really in putting together some helpful monitoring. Appreciate you taking the time to write these up. 

Thank you!

Marlan

[CoreyS]

We're happy you find this helpful @Marlan !

[reneege]

Thank you much for this info, which helps us to understand the audit log details better!  Hope Dataiku will make an enhancement in near future to add a summary dashboard/report for user activities per projects under 'Administration -> Maintenance".

[Turribeach]

I second this request, Dataiku should provide better data on how users are using the app. Audit logs are not really suited for that although they can be used.