Optimization for runtime

Mankaran119
Mankaran119 Registered Posts: 2 ✭✭✭

I have a flow with a lot of recipes - python/SQL/SPARK & snowflake datasets, with multiple scenarios created & optimized engine used.

Still my flow takes 3 hours to run & I am looking for ways to reduce this runtime, any tips & methods anyone can suggets


Operating system used: Windows

Answers

  • Manuel
    Manuel Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 193 ✭✭✭✭✭✭✭

    Hi,

    It is hard to suggest actions without looking at the specific flow, but here are some generic suggestions:

    Other than this, contact your Dataiku Customer Success Manager, they can help you engage Dataiku services to optimise your flow.

    I hope this helps.

    Best regards

Setup Info
    Tags
      Help me…