Issue: Non-matching partitioning scheme for ยซTable Nameยป R recipe

abhiboss25
Level 1
Issue: Non-matching partitioning scheme for ยซTable Nameยป R recipe

Hi Experts,

I have a R recipe and I am writing to data frames (which are hive tables) using below function.

dkuWriteDataset(df1,"Hive Table A") - for this I want partition pattern as %Y/%M/%D/.*

Partition_pattern_A.jpg

โ€ƒdkuWriteDataset(df2,"Hive Table B") - for this I want partition pattern as %Y/%M/%D/%H/.*

Partition_pattern_B.jpg
but while running recipe it gives me error: Non-matching partitioning scheme for Hive Table B.
However, if I keep partition pattern same for both the tables ( %Y/%M/%D/.*) then there is no issue.

Also, I would like to know how can I create partition for every run?

0 Kudos
2 Replies
Liev
Dataiker Alumni

Hi @abhiboss25 

Not sure if you managed to solve the issue on your own in the end. 

When dealing with partitions, it is important to be clear on the partitions to be built and the partitions to use in order to build them. 

Please refer to our documentation on this topic.

Good luck!

0 Kudos
Clรฉment_Stenac

Hi,

Specifically, what you are trying to do is impossible. Dataiku cannot write, in a single recipe, to two datasets that don't have the same partitioning scheme.

Hence, you cannot write to a day-partitioned and to a hour-partitioned dataset at the same time.

You will need to use two recipes for that.

0 Kudos