Error while running a script step - 'ArrayFold' the first rows already used 500 MB

clayms Registered Posts: 52 ✭✭✭✭

I get the following error when trying to run a Clean recipe with an ArrayFold as a step. The recipe says it completes without error, however the resulting dataset clearly does not have the specified columns.

How can I get around this?

Error while running a script step

When processing 'ArrayFold', the first 2697 rows already used 500 MB

Additional technical details: HTTP code: 400, Code: STEP_RUN_EXCEPTION, type: ApplicativeException

Operating system used: centos


  • Emma
    Emma Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer Posts: 52 Dataiker

    Hey @clayms

    Do you mean that you see this error when you are within the Prepare recipe view (before you click Run)? If so, it is because you have hit the memory limit for displaying the sample and you have a few options...

    1. You can try to increase the size limit for processed samples, either in the Sample Settings pane on the left of the script OR in the Administration > Settings > Resource Control > Explore / Prepare sample limit.
    2. Run the recipe (even though the View in the Prepare Recipe Script does not populate).

    If you continue to experience problems please open a support ticket with the logs of your problem so that our team can assist you,


Setup Info
      Help me…