Dataiku Python API : create a plugin recipe

Options
MGad
MGad Registered Posts: 14 ✭✭✭✭

Hello

I started looking into Dataiku API, to create predefined subflows triggered via a macro (previous discussion here : https://community.dataiku.com/t5/Plugins-Extending-Dataiku-DSS/Automatically-create-predefined-flows-with-a-plugin/m-p/8081#M433)

I found this documentation section on python methods to create recipes : https://doc.dataiku.com/dss/latest/python-api/rest-api-client/reference.html#recipes

However, it seems to be limited to native Dataiku recipes.

Can I create a custom recipe, from a plugin, using Dataiku's python API ?

Thx for your help

Answers

  • Andrey
    Andrey Dataiker Alumni Posts: 119 ✭✭✭✭✭✭✭
    edited July 17
    Options

    Hi,

    Yes, it's possible to create recipes using Python API even if they're defined in plugins.

    Here's an example:

        creator = dataikuapi.dss.recipe.DSSRecipeCreator("CustomCode_active-learning-query-sampler","query-sampler",p)
        creator.with_input(saved_model_id, role='saved_model')
        creator.with_input('j47pTfgK', role='unlabeled_samples')
        creator.with_output('image-queries', role='queries')
    
        creator.recipe_proto = {
          "type": "CustomCode_active-learning-query-sampler",
          "neverRecomputeExistingPartitions": False,
          "optionalDependencies": False,
          "params": {
            "customConfig": {
              "batch_size": 1,
              "confidence": 0.5,
              "gpu_allocation": 1,
              "list_gpu": "0",
              "record_missing": False,
              "strategy": "confidence",
              "should_use_gpu": True
            },
            "containerSelection": {
              "containerMode": "INHERIT"
            }
          },
          "customMeta": {
            "kv": {}
          },
          "redispatchPartitioning": False,
          "maxRunningActivities": 0,
          "inputs": {
            "saved_model": {
              "items": [
                {
                  "ref": saved_model_id,
                  "deps": []
                }
              ]
            },
            "unlabeled_samples": {
              "items": [
                {
                  "ref": "j47pTfgK",
                  "deps": []
                }
              ]
            }
          },
          "outputs": {
            "queries": {
              "items": [
                {
                  "ref": "image-queries",
                  "appendMode": False
                }
              ]
            }
          }
        }
        
        creator.set_raw_mode()
        creator.create()

    In order to obtain the correct recipe prototype you can first create a recipe manually, then call this snippet to obtain a list of recipes and take it from there :

    import dataiku, dataikuapi
    c = dataiku.api_client()
    p = dataikuapi.dss.project.DSSProject(c, dataiku.Project().project_key)
    p.list_recipes()

    Regards

Setup Info
    Tags
      Help me…