Sort Recipe is useless

Solved!
davidmakovoz
Sort Recipe is useless

I'm trying to apply the Sort recipe to a dataset with ~18M rows and 134 columns. The input and output datasets are in a database and the recipe is supposed to run in-database (SQL).  I tried different columns to sort by: date, integer, string. I'm always getting this warning message

Warnings were encountered

  • WARNING WARN_RECIPE_SORT_OUTPUT_DS_ORDER_NOT_SUPPORTED
    Output dataset is not compatible with order preservation
    The output dataset type does not support order preservation, it seems that the sort recipe will be useless. More information available in the doc.

I didn't find anything relevant in the documentation.

 

Please, help.

0 Kudos
1 Solution
Ignacio_Toledo

Hi @davidmakovoz, in what kind of dataset are you storing the output? There are some hints at https://doc.dataiku.com/dss/latest/other_recipes/sort.html#write-ordering, and if your output dataset doesn't preserve order writing, you get this message, becase the problem is not the Recipe nor DSS, but the database that is getting the output.

View solution in original post

0 Kudos
3 Replies
Ignacio_Toledo

Hi @davidmakovoz, in what kind of dataset are you storing the output? There are some hints at https://doc.dataiku.com/dss/latest/other_recipes/sort.html#write-ordering, and if your output dataset doesn't preserve order writing, you get this message, becase the problem is not the Recipe nor DSS, but the database that is getting the output.

0 Kudos
Samashti
Level 1

Hi Ignacio,

I  am facing the same error but going and fixing the output settings as laid out in the write ordering document is also not helping. could you please detail down the steps to follow?

0 Kudos
Ignacio_Toledo

Hi @Samashti 

What kind of connection (or dataset) are  you storing the result of the recipe? A filesystem, HDFS, SQL, S3, etc?

Cheers

Ignacio

0 Kudos