Order not maintained in nesting columns

dataiku_user Registered Posts: 7 ✭✭✭✭

I have several columns that I am combining using data preparation recipe - and the 'nest' functionality. A number of these columns are in pairs and I am nesting them into a third column.

E.g: a1,b1 nested (in that order) to a1_b1, a2,b2 nested to a2_b2 and so on. Thus the map object created should be a1 and then b1.

However, an issue I face is for certain column pairs, the ordering is not maintained. i.e. when nested, the b column comes before a though the selection is first a and then b. The issue seems to affect in a random fashion.


a b
1 2


{"b": 2, "a": 1}

Is this a bug or am I missing something?

Best Answer

  • Alex_Combessie
    Alex_Combessie Alpha Tester, Dataiker Alumni Posts: 539 ✭✭✭✭✭✭✭✭✭
    Answer ✓

    When using the "Nest columns" processor in a visual Prepare recipe, DSS converts the columns to a key:value format. By design, this format is not ordered by key. If you want to preserve order, I would advise using a "Concatenate" processor.

    Note that for advanced processing needs, the visual Prepare recipe supports a formula language close to Excel, and base Python.




Setup Info
      Help me…