Sort Recipe is useless

davidmakovoz

I'm trying to apply the Sort recipe to a dataset with ~18M rows and 134 columns. The input and output datasets are in a database and the recipe is supposed to run in-database (SQL). I tried different columns to sort by: date, integer, string. I'm always getting this warning message

Warnings were encountered

• WARNING WARN_RECIPE_SORT_OUTPUT_DS_ORDER_NOT_SUPPORTED

  Output dataset is not compatible with order preservation

  The output dataset type does not support order preservation, it seems that the sort recipe will be useless. More information available in the doc.

I didn't find anything relevant in the documentation.

Please, help.

Ignacio_Toledo

Hi @davidmakovoz
, in what kind of dataset are you storing the output? There are some hints at https://doc.dataiku.com/dss/latest/other_recipes/sort.html#write-ordering, and if your output dataset doesn't preserve order writing, you get this message, becase the problem is not the Recipe nor DSS, but the database that is getting the output.

Samashti

Hi Ignacio,

I am facing the same error but going and fixing the output settings as laid out in the write ordering document is also not helping. could you please detail down the steps to follow?

Ignacio_Toledo

Hi @Samashti

What kind of connection (or dataset) are you storing the result of the recipe? A filesystem, HDFS, SQL, S3, etc?

Cheers

Ignacio