Community Conundrum 25: Feature Visualization is now live! Read More

Issue: Non-matching partitioning scheme for «Table Name» R recipe

Level 1
Issue: Non-matching partitioning scheme for «Table Name» R recipe

Hi Experts,

I have a R recipe and I am writing to data frames (which are hive tables) using below function.

dkuWriteDataset(df1,"Hive Table A") - for this I want partition pattern as %Y/%M/%D/.*

Partition_pattern_A.jpg

 dkuWriteDataset(df2,"Hive Table B") - for this I want partition pattern as %Y/%M/%D/%H/.*

Partition_pattern_B.jpg
but while running recipe it gives me error: Non-matching partitioning scheme for Hive Table B.
However, if I keep partition pattern same for both the tables ( %Y/%M/%D/.*) then there is no issue.

Also, I would like to know how can I create partition for every run?

0 Kudos
2 Replies
Dataiker
Dataiker

Hi @abhiboss25 

Not sure if you managed to solve the issue on your own in the end. 

When dealing with partitions, it is important to be clear on the partitions to be built and the partitions to use in order to build them. 

Please refer to our documentation on this topic.

Good luck!

0 Kudos
Dataiker
Dataiker

Hi,

Specifically, what you are trying to do is impossible. Dataiku cannot write, in a single recipe, to two datasets that don't have the same partitioning scheme.

Hence, you cannot write to a day-partitioned and to a hour-partitioned dataset at the same time.

You will need to use two recipes for that.

0 Kudos