Replace function not working properly in Prepare recipe
UserBird
Dataiker, Alpha Tester Posts: 535 Dataiker
Hi All,
What I am trying to do is replace empty cells with '0'. After running the recipe, only a few rows are getting replaced with 0 and others remain blank.
Please note that this is only happening when I select Spark engine. Other engines do not cause this problem. Is this a known problem with spark engine in prepare recipe?
What I am trying to do is replace empty cells with '0'. After running the recipe, only a few rows are getting replaced with 0 and others remain blank.
Please note that this is only happening when I select Spark engine. Other engines do not cause this problem. Is this a known problem with spark engine in prepare recipe?
Tagged:
Answers
-
Hi,
We confirm this as a known issue when using "All" or "Pattern" columns selection mode in Spark engine: filling of value will only start after the first non-empty value. We are working on this. -
Are there any updates on this or work arounds?
-
All - is there any workarounds? I tried running it in both local stream and spark, and it didnt work (i.e. only a few rows were replaced with '0')
-
CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭@adamnieto
@GSung
Thank you for following up. We can confirm that there are no updates or any recommended workarounds at this point. We are still working on this and will take your questions and feedback into consideration.
