Taking much time for scenario

Setup Info
Registered Posts: 18 ✭✭✭

Hi team,

I created the pipeline and set the scenario for it, but it takes soooo much time to run the scenario.

Once I clicked "run" button, green triangle mark("▶" in green) is shown in a specific dataset for over 20 minutes…

Does this mean the dataset has so much updated data? or just any mis-setting on scenario?

Answers

  • Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,326 Neuron

    Scenarios don't add runtime to your jobs so this is a problem with your job not the scenario. Look at the job log to figure out what's happening.

  • Partner, PartnerApplicant, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 12 Partner

    Please check the target recipe from the following perspectives:
    - The execution engine is not labeled "slow.
    - The data source to be processed is not more than a few GB.
    - In the case of partitioning, is there any unnecessary duplication of calculations?
    - Is there any unnecessary duplication of calculations in case of partitioning?

Welcome!

It looks like you're new here. Sign in or register to get started.

Welcome!

It looks like you're new here. Sign in or register to get started.