Dataiku Python API : create a plugin recipe

MGad · July 2020

Hello

I started looking into Dataiku API, to create predefined subflows triggered via a macro (previous discussion here : https://community.dataiku.com/t5/Plugins-Extending-Dataiku-DSS/Automatically-create-predefined-flows-with-a-plugin/m-p/8081#M433)

I found this documentation section on python methods to create recipes : https://doc.dataiku.com/dss/latest/python-api/rest-api-client/reference.html#recipes

However, it seems to be limited to native Dataiku recipes.

Can I create a custom recipe, from a plugin, using Dataiku's python API ?

Thx for your help

Andrey · July 2020

Hi,

Yes, it's possible to create recipes using Python API even if they're defined in plugins.

Here's an example:

    creator = dataikuapi.dss.recipe.DSSRecipeCreator("CustomCode_active-learning-query-sampler","query-sampler",p)
    creator.with_input(saved_model_id, role='saved_model')
    creator.with_input('j47pTfgK', role='unlabeled_samples')
    creator.with_output('image-queries', role='queries')

    creator.recipe_proto = {
      "type": "CustomCode_active-learning-query-sampler",
      "neverRecomputeExistingPartitions": False,
      "optionalDependencies": False,
      "params": {
        "customConfig": {
          "batch_size": 1,
          "confidence": 0.5,
          "gpu_allocation": 1,
          "list_gpu": "0",
          "record_missing": False,
          "strategy": "confidence",
          "should_use_gpu": True
        },
        "containerSelection": {
          "containerMode": "INHERIT"
        }
      },
      "customMeta": {
        "kv": {}
      },
      "redispatchPartitioning": False,
      "maxRunningActivities": 0,
      "inputs": {
        "saved_model": {
          "items": [
            {
              "ref": saved_model_id,
              "deps": []
            }
          ]
        },
        "unlabeled_samples": {
          "items": [
            {
              "ref": "j47pTfgK",
              "deps": []
            }
          ]
        }
      },
      "outputs": {
        "queries": {
          "items": [
            {
              "ref": "image-queries",
              "appendMode": False
            }
          ]
        }
      }
    }
    
    creator.set_raw_mode()
    creator.create()

In order to obtain the correct recipe prototype you can first create a recipe manually, then call this snippet to obtain a list of recipes and take it from there :

import dataiku, dataikuapi
c = dataiku.api_client()
p = dataikuapi.dss.project.DSSProject(c, dataiku.Project().project_key)
p.list_recipes()

Regards

Dataiku Python API : create a plugin recipe

Answers

Categories

Setup Info

Tags