No Logging "DEBUG INFO Message " in the log file at project

NerGuRy

hello.
I would like to inquire about log settings.
“Jobs” and “Scenarios” are being performed every minute in my dataiku project.
I am keeping one week's worth of logs, but there is "[02:09:41] [INFO] [dku.utils] - [2023-11-20 02:09:40,211] [73/MainThread] [DEBUG] " in the log
Because there are a lot of debug INFO logs, 3M files are created every minute of execution, which means a lot of storage usage.
I would like to reduce storage usage by loading only error logging without saving the debug INFO log.
Please tell me where I can set this.
thank you.

---

Operating system used: linux

---

Operating system used: linux

Alexandru

Hi @NerGuRy
,

Reducing the verbosity of logs can affect your ability to support you if you have job failures or issues.
So we would recommend staying on the default log levels. Moving to ERROR only would hinder the ability to support you.

The output.log is not the only file created whenever a job is run. A snapshot of the project configuration is also performed so it can be referenced during an investigation of a potential failure.

If you have any particular very large output logs files created please share those job diagnostics over a support to investigate.

Kind Regards,

Turribeach

I agree with Alex. Rather than modifying the verbosity of the logs I would suggest you increase the purging of old logs for those projects that run high frequency of jobs. Have a look at the project macros that can help you with you with this. You can schedule project macros in a scenario as well.

NerGuRy

Thanks for yor help.

I understood what you mean, and I added JOB which process deleting log file.

Have a Good Day^^

NerGuRy

Thanks for yor help.

I will check Macro Func.

Have a Nice Day ^^