Free up storage space on Dataiku

[rafael_rosado97]

Hi, everyone.

I want  to free up storage space on Dataiku disk. I was reading this documentation: https://doc.dataiku.com/dss/latest/operations/disk-usage.html.

So I want to eliminate Job logs through Macro button. At this point I saw another Macro called clear continuous activities logs. What is the difference between these two Macros?

Is it recommended to elimanate these logs in a project in which I'm working currently or only in those that I have finished? I ask this because, what would happen if I want to modify an old proyect in the future?; Will it affect?

Any other option to free up storage space?

Thanks!

[JordanB]

Hi @rafael_rosado97,

Job logs consist of historical data and are useful for understanding job issues. It is safe to remove folders of jobs that are not currently running. Therefore, you can safely run the macro in a project you're currently working on as long as no jobs are running at that time. Once you run the macro, the logs of these jobs will not be available anymore, but the existence of the job will still be registered in the DSS UI.

Continuous activity logs are specific to continuous activity recipes: https://doc.dataiku.com/dss/latest/streaming/concepts.html#continuous-recipe

Job logs encompass all of the various other types of jobs and you can specify a max age. For example, below, logs 15 days and older will be cleared.

Please let me know if you have any further questions.

Thanks!

Jordan

[rafael_rosado97]

It's very clear, @JordanB. Thank you so much.

Just one more question: how can I visualize the available store on the Dataiku disk? I execute the command df -h on the terminal but I would like global information (total space used). Is it possible?

[JordanB · Accepted solution]

Hi @rafael_rosado97,

That is the command we would use to display info about total space and available space on a file system.

Can you perhaps explain in more specific terms what you are looking for?

Thanks,

Jordan