How to assign the result of a recipe to the datasets used as input?

Options
rafael_rosado97
rafael_rosado97 Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 58 Partner

Hello, everyone.

Imagine the next situation:

1. I make a query with a SQL recipe to a dataset that is on a db where I get 10 days.

2. Next day, these 10 days change their values, so I want to make the query again to get the new values and remplace the old ones. But I want just to maintain the first day of the old ones. It is because this day is kept as DB history.

The next day is the same thing. For that reason I want to find a way to remplace the same input for the result of the output and making like a data history. Like a recursive.

Edit:

The result is saved on Big Query.

The idea is not take the history because it will increase over time. So it could be like a DELETE and INSERT SQL statements. Delete repeated old 10 days and insert those days but with new values. Remember to keep just the first day. Is it possible?

Could you give ideas to solve it?

Thank you!!!!

Tagged:

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,727 Neuron
    Options

    What you are trying to do makes no sense to me, since you are not really describing your requirement but how you think you can achieve it. Also the subject of the post makes no sense with the body of the question. So I will answer your question, but I suspect you are not doing things in the most optimal way.

    If you want to keep 1 specific day from the 10 days of data you could use a different dataset. Then do a ranking and get only the first or day of the 10 days on a separate dataset. Again this makes no sense to me so it's hard to suggest a solution without understanding what the requirement is.

    >> How to assign the result of a recipe to the datasets used as input?

    It depends on the recipe type. In Python you will use the input directly in code. In a visual recipe you use it in a Join recipe.

Setup Info
    Tags
      Help me…