Partitioning - Flow epoch

Options
dadbuz
dadbuz Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer Posts: 16 ✭✭✭✭

Can someone please point me to the section in the documentation that explains/discuss `Flow epoch` purpose and how to use in the advanced settings of dataset - i can't find the relevant section in the online docs!

Also, I'm having issue in activating partitioning (time and discrete based dimensions) for my external teradata dataset. I get One of the partition values for [table] d is null or empty, unsupportedfor when for instance trying to sample from the dataset. i tried to tick the `Missing partitions as empty` and `Empty as not ready` features but doesn't seem to ignore the empty partitions or fix the issue.

Thanks

Answers

  • CoreyS
    CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭
    Options

    Hi @dadbuz
    while you wait for a more complete response, have you checked out both the Core Designer and Advanced Designer Learning Paths on the Dataiku Academy? The latter in particular should provide you more guidance on the visual tools of Dataiku DSS. I hope this helps!

  • dadbuz
    dadbuz Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer Posts: 16 ✭✭✭✭
    Options

    Hey @CoreyS
    , yes. I've indeed followed both courses plus the course on partitioning but no mention to the flow epoch feature. Thanks.

  • dadbuz
    dadbuz Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer Posts: 16 ✭✭✭✭
    Options

    Hi, any updates on this? thanks

  • Jason
    Jason Registered Posts: 29 ✭✭✭✭✭
    Options

    I know this is an older topic, so I'm reluctant to bring back up, but I had the same problem as the OP with regard to the error message on the partitioning values. Here was my fix:

    In my case, I had several tables that were coming together in a join. I had partitioned two of the input tables with the name of the column I wanted, but the problem was that one of these tables (the first in the join) used one column name, and the other table used a DIFFERENT column name, so the partitioning was failing on the second table due to the name of the column not being correct, but the error message doesn't give you any clue this is the case. Hope that helps anyone in the future that runs into this obscure error.

Setup Info
    Tags
      Help me…