How do I specify partitions?
UserBird
Dataiker, Alpha Tester Posts: 535 Dataiker
Best Answer
-
There are two places where you specify partitions in a recipe:
- when running the recipe, you need to tell DSS wich partitions of the output dataset you want to build. For instance “1789-07-14/1789-07-23” would ask DSS to build ten consecutive partitions (pertaining to a long time ago).
- when defining the recipe, you need to specify “partition dependencies”. Whenever DSS wants to build a partition of the output, it needs to know which partition of the input dataset are to be used. There are many ways to specify a dependency, for instance “from two days before to 0 days before”, in which case to build the output partition 1789-07-20 DSS would use the input partitions 1789-07-18 to 1789-07-20.
See also http://doc.dataiku.com/dss/latest/partitions/identifiers.html