Replace some values in a column with previous values from another column
Good day!
I am trying to replace some of the values in one column (empty ones) with values from another column but from previous row. Can't find a solution.
For example: I have columns "A" and "B". What should I do, if I need to replace empty cells A3, A7, A20 etc (where 3, 7, 20 are the numbers of rows from the top) with the values from B2, B6, B19 etc cells?
What would you recommend?
Best Answer
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,757 NeuronOptions
This can be done with a Window recipe. Assuming you have a dataset like this:
Create a Window recipe and set the Window Definition. You need to have a sort order to fetch the previous value so pick the column(s) that has your rows ordered properly:
Then set the aggregation to section to calculate the lag of the row (previous). You can also use the lead (next) if you want:
Run the recipe and you will have the output like this:
Now simply replace the value of A by B in a Prepare recipe using whatever logic you want.
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,757 NeuronOptions
Where is your dataset stored? (which technology). Are you looking for a coders solution (Python recipe) or a clickers solution (Visual recipe)?
-
ttoropov Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 8 ✭Options
Hello, right, sorry. Forgot to mention.
I'm working in a prepare recipe.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,757 NeuronOptions
Where is your dataset stored? (which technology)
-
ttoropov Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 8 ✭Options
Sorry, It's a Server's Filesystem (in csv format), if that's what you are asking for.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,757 NeuronOptions
Yes, thanks. I wanted to check if SQL was available as this will be much simpler in SQL.
-
ttoropov Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 8 ✭Options
Good idea! Thank you very much!
-
Thank you for the solution. It was exactly what I was looking for. It took me a while to find this page though.
