Create a Recipe via API which outputs to a managed Folder

dongari
dongari Registered Posts: 6 ✭✭✭✭
edited July 16 in Using Dataiku

Dear Dataiku Community,

it currently seems impossible to create via API a Recipe which outputs to a managed folder instead of a Dataset. See the following code:

crc = p.new_recipe(type='python', name="compute_B_from_A")
crc.with_input("A")

# Solution 1 not working
mf = p.create_managed_folder(name="B", connection_name='filesystem_folders')
crc.with_output(mf)

# Solution 2 not workin neither
crc.with_new_output_dataset(name="B",connection='filesystem_folders')

crc.with_script(crc3_code)
r = crc.create()
r.run(wait=True)    

Actually, I can do it using the graphical interface:

dataiku_folder.png

Please let me know if there is a correct way to do it via API!

Thanks!

Danieel


Operating system used: UNIX


Operating system used: Unix

Best Answer

  • HarizoR
    HarizoR Dataiker, Alpha Tester, Registered Posts: 138 Dataiker
    edited July 17 Answer ✓

    Hi Daniele,

    You almost had it in your Solution 1 The with_output() method of the DSSRecipeCreator takes as argument either the Dataset name or the Folder's identifier (see the documentation for more details).

    To apply this to your use-case, use the id attribute of the mf object like this:

    crc.with_output(mf.id)

    Note that programmatically creating Flows can be quite challenging, so you should tread with caution and make sure to implement guardrails at sensitive places (e.g. checking if a managed folder already exists before creating it).

    Hope this helps!

    Best,

    Harizo

Answers

  • dongari
    dongari Registered Posts: 6 ✭✭✭✭
    edited July 17

    SOLVED.
    Thanks! It worked well.
    I summarize the full working code for clarity:

    crc = p.new_recipe(type='python', name="compute_B_from_A")
    crc.with_input("A")
    mf = p.create_managed_folder(name="B", connection_name='filesystem_folders')
    crc.with_output(mf.id)
    crc.with_script("print('hello world!')")
    r = crc.create()
Setup Info
    Tags
      Help me…