DSS EventServer - How to clean old files generated by the EventServer ?

rona Registered Posts: 47 ✭✭✭✭✭


We activated the DSS EventServer in order to get some KPI related to the DSS API Node endpoints execution.

We would like to implement a process to clean regularly all the files older than 30 days.

Please, do you know the best approach to setup this process ?

The log files are currently stored with a “filesystem-like” connection.

Best Answer

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 Dataiker
    Answer ✓


    While there are currently no built-in macros to manage this from DSS.

    If you are using S3 or similar you can use S3 lifecycle policies to remove data older than x days.

    Otherwise, a simple bash script and cron job to clean up the respective files regularly feel like the easiest approach.

    e.g find /path_to_/topic/routing-key/* -mtime +30 -exec rm {} \;

Setup Info
      Help me…