Access partitions

lisa811
lisa811 Partner, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Registered Posts: 14 Partner

Hello,



I don't get how to "access" the different partitions dataiku created. I've got the dataset partions (see picture) which is divided by in my case station_uuid, but in the resulting dataset this column is missing. It also looks like there are different folders created. How can I access the partitions separately?







Thanks

Tagged:

Answers

  • jereze
    jereze Alpha Tester, Dataiker Alumni Posts: 190 ✭✭✭✭✭✭✭✭

    Hi,

    When you open and see a dataset, you can configure the sampling to display only one partition.

    • On a dataset explore, click "Configure sample"
    • In "Use partitions", choose "select partitions"
    • In "partitions", select or enter the value of the partition you want to explore
  • lisa811
    lisa811 Partner, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Registered Posts: 14 Partner
    Thanks a lot!
    One more question: Now I would like to run a recipe on all of the partitions (clean forecast recipe). A colleague of yours recommended a variable. So I created a variable called uuid_list, which contains all the partitions separated by a "/". But I don't get how to use this now (see picture with the error above). Is there any other solution which helps me to apply the recipe to each of the partitions?
    Thanks
  • lisa811
    lisa811 Partner, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Registered Posts: 14 Partner
    Sorry the picture was missing
  • jereze
    jereze Alpha Tester, Dataiker Alumni Posts: 190 ✭✭✭✭✭✭✭✭
    Variable expansion works when using the Build/Train steps in scenarios, not in recipes.
  • lisa811
    lisa811 Partner, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Registered Posts: 14 Partner
    Ok so it's not possible to apply a recipe to all partitions simultaneously?
  • jereze
    jereze Alpha Tester, Dataiker Alumni Posts: 190 ✭✭✭✭✭✭✭✭
    You have two ways to do it today:

    1) You list the partitions in the "Recipe run options" (without using a variable). Note that your browser will remember the previously entered value.
    2) You build your flow/recipes using a scenario (and you can use a manual trigger).

    I noted your interest in the variable expansion system in the "Recipe run options".
  • lisa811
    lisa811 Partner, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Registered Posts: 14 Partner
Setup Info
    Tags
      Help me…