Join Partitioned Datasets

ojhaabhi17
ojhaabhi17 Registered Posts: 1 ✭✭✭

I would like to join two partitioned datasets (both date partitioned), by creating week number (not created in flow)

Dataset1 is partitioned on execution dates(weekly once) of year 2019.
Dataset2 is partitioned on execution dates(weekly once) of year 2018.
I want to join these two datasets by week number of year (not present in the flow).
In join recipe I thought of creating week number of execution date/partitioned date (thru "pre-join computed columns" section), but not succeeded.

Could you help me out.
Thanks

Answers

  • HarizoR
    HarizoR Dataiker, Alpha Tester, Registered Posts: 138 Dataiker

    Hi ojhaabhi17,

    The easiest way to retrieve the week numbers is to insert an "Extract date components" step in an upstream Prepare recipe for each of the Datasets you want to join. You may need to create those recipes if they don't exist.

    From there, you have the ability to extract a number of date elements into new columns, including the "Week of the year" information. Once you have done that, you will have your join key columns.

    Screenshot 2021-08-25 at 09.00.56.png

    Best,

    Harizo

Setup Info
    Tags
      Help me…