Filter recipe : How to avoid stop processing when there are no matched records

databirds11

Hi dataiku users,

I want to know how to resolve the situation in subject.

I use filter recipe only for processing the exception data, and stack with main data after that, so if there are no filtered records in output dataset, no problem.

but in dataiku, if there are not all data sets in stack recipe, return error and stop processing in that time.

Do you know how to ?

databirds11

I could avoid error by changing engine of filter recipe, "Spark" to "DSS".
It seems DSS engine allow to create 0 record dataset, but Spark engine is not.

Turribeach

Have a read at this post which basically does exactly what you want which is to conditionally control the execution of the scenario steps based on some data:

https://community.dataiku.com/t5/What-s-New/Want-to-Control-the-Execution-of-Scenario-Steps-With-Conditional/ba-p/31472

Turribeach

If you are using Databricks this is a bug. Raise it with Dataiku Support.