Error while running a script step

GSung
GSung Registered Posts: 27 ✭✭✭✭

I have uploaded a huge dataset onto Dataiku and I have tried to concatenate two columns but encountered below error message. How do I troubleshoot this error message?

thank you!

Error while running a script step

When processing 'ColumnsConcat', the first 118842 rows already used 500 MB

Answers

  • fchataigner2
    fchataigner2 Dataiker Posts: 355 Dataiker

    Hi

    this means you are working on a too large sample of the input data in your Prepare recipe. You should go in the sampling settings (on the left edge of the screen) and reduce the number of rows. Note that when running the recipe, the full dataset will be used, not just the sample used for designing the steps in the UI

  • Ankur5289
    Ankur5289 Partner, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Registered Posts: 27 Partner

    @fchataigner2
    where do we get sampling settings in a prepare receipe. we see sampling setting only on a specific data set.

Setup Info
    Tags
      Help me…