Executing a prepare recipe gives a blank dataset
Hi, today doing a recipe on my workflow (it was about removing columns from the dataset,removing rows with empty values in all columns and replace specific values in 1 column) after creating the script, the preview show the dataset with the script implemented, obtaining an expected result (from a dataset with 217 rows 52 columns to 63 rows and 35 columns). After saving and running the recipe with local stream the resulting dataset is empty, which is bit weird. I tried also running with spark and nothing changed.
Best Answer
-
Jesus Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 8 ✭✭✭✭Options
Thank you for the help, seems that when i split up the process in two parts it works. The problem was on the remove rows with empty values in all columns, just if you wonder.
Answers
-
Manuel Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 193 ✭✭✭✭✭✭✭OptionsHI Jesus,
My tip for these cases is to try it with different engines, but it seems you already tried that.
To debug, my suggestion is that you toggle each of the prepare steps off (each processor has an on/off icon) to identify the culprit. Then you can analyse in more detail what is happening.
I hope this helps.
Best regards