Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I have a R recipe and I am writing to data frames (which are hive tables) using below function.
dkuWriteDataset(df1,"Hive Table A") - for this I want partition pattern as %Y/%M/%D/.*
dkuWriteDataset(df2,"Hive Table B") - for this I want partition pattern as %Y/%M/%D/%H/.*
but while running recipe it gives me error: Non-matching partitioning scheme for Hive Table B.
However, if I keep partition pattern same for both the tables ( %Y/%M/%D/.*) then there is no issue.
Also, I would like to know how can I create partition for every run?
Specifically, what you are trying to do is impossible. Dataiku cannot write, in a single recipe, to two datasets that don't have the same partitioning scheme.
Hence, you cannot write to a day-partitioned and to a hour-partitioned dataset at the same time.
You will need to use two recipes for that.