add an incremental column in dataset

deep_215

requirement is to add an incremental column in datset, it should not be an identity column however data in it will be unique.

Manuel

Hi,

You can use a Window recipe for this:

• In the Window definition, enable a order column
• In the Aggregations definition, compute the Row Number
• See attached image

For an overview of the Window recipe, check :

I hope this helps.

yesitsmeoffical

is it possible to do this in the prepare recipe? not that I don't know how to use window but would like to keep all changes consolidated in a single recipe if possible

Brennan

Using a python code step in a prepare recipe:

count = 0
def process(row):
    global count
    count += 1
    return count

Umut

https://community.dataiku.com/discussion/23256/add-an-incremental-column-in-dataset

Hi,

You can use rowIndex function with Formula processor in the Prepare recipe. I just added a sample screenshot to show how this can be calculated.

I hope this helps,

Best,

Muhammad Farhan

https://community.dataiku.com/discussion/comment/44874#Comment_44874

Hi Umut

This is a great solution as I can now use it with my Spark engine in prepare recipe. But may I know how do you figure out the use of rowIndex? Was it documented anywhere?