Best settings of Flush size and interval for Event Server
Hi! Dataiku,
Recently, I installed Event Server and got log files from DSS nodes, thanks to it.
and tried to search Flush Size and Flush Interval from Dataiku Documentation, but there's nothing.
so, I have no idea how much I have to input values for them.
Can you give me the best example settings for Flush Size and Flush Interval?
Operating system used: Linux
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,226 Dataiker
Hi @Nake
,So for the "Flush size (bytes)" and "Flush interval (seconds)" parameters.
The default values are a good starting point and should be adjusted based on your needs.
Lower values will make the event server produce many small files with only a few events, which is often not desirable. Larger values will create fewer bigger files, but if the event server crashes, potentially more events are lost.
You can monitor and see how many logs your instances produce and adjust based on your usage of the logs and the frequency you want them.