How to apply a partition-trained model to all partitions

Options
maxData
maxData Registered Posts: 1 ✭✭✭

Hello,

I am trying to train a partitioned model on all partitions of a dataset and then apply this model to make predictions on another partitioned dataset.

Dataiku presents an option to train on 'all partition' but when I try to score, using the 'all available' partitions parameter, it fails (I get the error "Path does not exist in the dataset: '/all available/' ").

So I am wondering if there is a way to make predictions on all the partitions of my dataset, without specifying them manually cause there will be too many.

Thanks in advance,

Max

Tagged:

Answers

  • pmasiphelps
    pmasiphelps Dataiker, Dataiku DSS Core Designer, Registered Posts: 33 Dataiker
    edited July 17
    Options

    Hi,

    You can use this python code to list all partitions - then copy paste into the partition selection box:

    partition_list = dataiku.Dataset("INPUT_DATASET_NAME").list_partitions()
    partition_list = '/'.join(partition_list)
    
    print(partition_list)

    Best,

    Pat

Setup Info
    Tags
      Help me…