Issue: Non-matching partitioning scheme for «Table Name» R recipe
Hi Experts,
I have a R recipe and I am writing to data frames (which are hive tables) using below function.
dkuWriteDataset(df1,"Hive Table A") - for this I want partition pattern as %Y/%M/%D/.*
dkuWriteDataset(df2,"Hive Table B") - for this I want partition pattern as %Y/%M/%D/%H/.*
but while running recipe it gives me error: Non-matching partitioning scheme for Hive Table B.
However, if I keep partition pattern same for both the tables ( %Y/%M/%D/.*) then there is no issue.
Also, I would like to know how can I create partition for every run?
Answers
-
Hi @abhiboss25
Not sure if you managed to solve the issue on your own in the end.
When dealing with partitions, it is important to be clear on the partitions to be built and the partitions to use in order to build them.
Please refer to our documentation on this topic.
Good luck!
-
Hi,
Specifically, what you are trying to do is impossible. Dataiku cannot write, in a single recipe, to two datasets that don't have the same partitioning scheme.
Hence, you cannot write to a day-partitioned and to a hour-partitioned dataset at the same time.
You will need to use two recipes for that.
