Accessing audit log with API

MRvLuijpen

Hello Community,

We were wondering if it would be possible to access the information inside the audit.log files with the use of Dataiku API. We are not able (security/DevOps doesn't allow this) to make a file connection to the run/audit files.

We have done some analysis on the audit log files (by downloading/uploading 2,1 GB logfiles) and we found several very useful information in these files.

We would really be helped with some sample code.

Thanks in advance,

Marc Robert

RoyE

Hello,

Thank you for the confirmation. Unfortunately, it is currently not possible to read the audit logs through the API but our engineers are aware of this request.

Audit logs are rotated at 100MB and up to 20 files are saved before rotated. However, you can modify the storage of the log location and the rotation described, here, to a location where you are able to read the files either through DSS or through python. Once this is done, you will not have to worry about DSS removing files, and you will be able to control how often these files are updated / accessed.

An alternative situation that you may want to look into is Event Server described, here.

Sincerely,

Roy

RoyE

Hello Marc,

Just for clarification, do you mean your security team does not want you to create an DSS internal managed dataset that is pointing to your <DATA_DIR>/run/audit folder? If so, do you happen to know the reason for this?

If you are able to do the above, please follow the steps below.

First, you will need to set up a connection that is pointing your data directory.

Once set up, you can create a filesystem that points to your /run/audit/ folder path to create a dataset based of your audit logs.

 

Unfortunately, we do not have have any APIs that are currently able to access the audit logs.

Roy

MRvLuijpen

Hi Roy,

Thanks for you reply. And you are right.

"The actual reason for the security team to not wanting to create an DSS internal managed dataset from the <DATA-DIR>/run/audit folder is that we/they do not know how, what and when Dataiku will lock these logging. And for the security process it is more important to have a complete audit log instead of read accessing the audit logs. "

With kind regards

MRvLuijpen

Thank you for this update