Sort Recipe is useless
![davidmakovoz](https://us.v-cdn.net/6038231/uploads/lithium_attachments/n1950i3778D985DEA99338.jpg)
I'm trying to apply the Sort recipe to a dataset with ~18M rows and 134 columns. The input and output datasets are in a database and the recipe is supposed to run in-database (SQL). I tried different columns to sort by: date, integer, string. I'm always getting this warning message
Warnings were encountered
- WARNING WARN_RECIPE_SORT_OUTPUT_DS_ORDER_NOT_SUPPORTED
Output dataset is not compatible with order preservation
The output dataset type does not support order preservation, it seems that the sort recipe will be useless. More information available in the doc.
I didn't find anything relevant in the documentation.
Please, help.
Best Answer
-
Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 412 Neuron
Hi @davidmakovoz
, in what kind of dataset are you storing the output? There are some hints at https://doc.dataiku.com/dss/latest/other_recipes/sort.html#write-ordering, and if your output dataset doesn't preserve order writing, you get this message, becase the problem is not the Recipe nor DSS, but the database that is getting the output.
Answers
-
Hi Ignacio,
I am facing the same error but going and fixing the output settings as laid out in the write ordering document is also not helping. could you please detail down the steps to follow?
-
Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 412 Neuron
Hi @Samashti
What kind of connection (or dataset) are you storing the result of the recipe? A filesystem, HDFS, SQL, S3, etc?
Cheers
Ignacio