Error while running a script step - 'ArrayFold' the first rows already used 500 MB

clayms
clayms Registered Posts: 52 ✭✭✭✭
in Using Dataiku

I get the following error when trying to run a Clean recipe with an ArrayFold as a step. The recipe says it completes without error, however the resulting dataset clearly does not have the specified columns.

How can I get around this?

Error while running a script step

When processing 'ArrayFold', the first 2697 rows already used 500 MB

Additional technical details: HTTP code: 400, Code: STEP_RUN_EXCEPTION, type: ApplicativeException

Operating system used: centos

· Share on FacebookShare on Twitter

Answers

  • Emma
    Emma Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 52 Dataiker

    Hey @clayms
    ,

    Do you mean that you see this error when you are within the Prepare recipe view (before you click Run)? If so, it is because you have hit the memory limit for displaying the sample and you have a few options...

    1. You can try to increase the size limit for processed samples, either in the Sample Settings pane on the left of the script OR in the Administration > Settings > Resource Control > Explore / Prepare sample limit.
    2. Run the recipe (even though the View in the Prepare Recipe Script does not populate).

    If you continue to experience problems please open a support ticket with the logs of your problem so that our team can assist you,

    Emma

    · Share on FacebookShare on Twitter
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…