Join Partitioned Datasets
I would like to join two partitioned datasets (both date partitioned), by creating week number (not created in flow)
Dataset1 is partitioned on execution dates(weekly once) of year 2019.
Dataset2 is partitioned on execution dates(weekly once) of year 2018.
I want to join these two datasets by week number of year (not present in the flow).
In join recipe I thought of creating week number of execution date/partitioned date (thru "pre-join computed columns" section), but not succeeded.
Could you help me out.
Thanks
Answers
-
Hi ojhaabhi17,
The easiest way to retrieve the week numbers is to insert an "Extract date components" step in an upstream Prepare recipe for each of the Datasets you want to join. You may need to create those recipes if they don't exist.
From there, you have the ability to extract a number of date elements into new columns, including the "Week of the year" information. Once you have done that, you will have your join key columns.
Best,
Harizo