Want to Stop Rebuilding "Expensive" Parts of your Flow? Explicit Builds are the Answer!READ MORE

Optimization for runtime

Mankaran119
Level 1
Optimization for runtime

I have a flow with a lot of recipes - python/SQL/SPARK & snowflake datasets, with multiple scenarios created & optimized engine used.

Still my flow takes 3 hours to run & I am looking for ways to reduce this runtime, any tips & methods anyone can suggets


Operating system used: Windows

0 Kudos
1 Reply
Manuel
Dataiker
Dataiker

Hi,

It is hard to suggest actions without looking at the specific flow, but here are some generic suggestions:

Other than this, contact your Dataiku Customer Success Manager, they can help you engage Dataiku services to optimise your flow.

I hope this helps.

Best regards

 

0 Kudos