Ready for Dataiku 10? Try out the Crash Course on new features!GET STARTED

Optimization for runtime

Mankaran119
Level 1
Optimization for runtime

I have a flow with a lot of recipes - python/SQL/SPARK & snowflake datasets, with multiple scenarios created & optimized engine used.

Still my flow takes 3 hours to run & I am looking for ways to reduce this runtime, any tips & methods anyone can suggets


Operating system used: Windows

0 Kudos
1 Reply
Manuel
Dataiker
Dataiker

Hi,

It is hard to suggest actions without looking at the specific flow, but here are some generic suggestions:

Other than this, contact your Dataiku Customer Success Manager, they can help you engage Dataiku services to optimise your flow.

I hope this helps.

Best regards

 

0 Kudos
A banner prompting to get Dataiku DSS