Order not maintained in nesting columns
dataiku_user
Registered Posts: 7 ✭✭✭✭
I have several columns that I am combining using data preparation recipe - and the 'nest' functionality. A number of these columns are in pairs and I am nesting them into a third column.
E.g: a1,b1 nested (in that order) to a1_b1, a2,b2 nested to a2_b2 and so on. Thus the map object created should be a1 and then b1.
However, an issue I face is for certain column pairs, the ordering is not maintained. i.e. when nested, the b column comes before a though the selection is first a and then b. The issue seems to affect in a random fashion.
Inputs;
a b
1 2
Outputs:
{"b": 2, "a": 1}
Is this a bug or am I missing something?
Tagged:
Best Answer
-
Hello,
When using the "Nest columns" processor in a visual Prepare recipe, DSS converts the columns to a key:value format. By design, this format is not ordered by key. If you want to preserve order, I would advise using a "Concatenate" processor.
Note that for advanced processing needs, the visual Prepare recipe supports a formula language close to Excel, and base Python.
Cheers,
Alex
Answers
-
Ok, thanks for a prompt answer. I figured out that backslash works for escape in the formula recipe