Discover this year's submissions to the Dataiku Frontrunner Awards and give kudos to your favorite use cases and success stories!READ MORE

Create a Recipe via API which outputs to a managed Folder

Solved!
dongari
Level 2
Create a Recipe via API which outputs to a managed Folder

Dear Dataiku Community,

it currently seems impossible to create via API a Recipe which outputs to a managed folder instead of a Dataset. See the following code:

 

 

crc = p.new_recipe(type='python', name="compute_B_from_A")
crc.with_input("A")

# Solution 1 not working
mf = p.create_managed_folder(name="B", connection_name='filesystem_folders')
crc.with_output(mf)

# Solution 2 not workin neither
crc.with_new_output_dataset(name="B",connection='filesystem_folders')

crc.with_script(crc3_code)
r = crc.create()
r.run(wait=True)    

 

 

 

Actually, I can do it using the graphical interface:

 

dataiku_folder.png

 

Please let me know if there is a correct way to do it via API!

Thanks!

Danieel


Operating system used: UNIX


Operating system used: Unix

0 Kudos
1 Solution
HarizoR
Developer Advocate
Developer Advocate

Hi Daniele,

You almost had it in your Solution 1 🙂 The with_output() method of the DSSRecipeCreator takes as argument either the Dataset name or the Folder's identifier (see the documentation  for more details).

To apply this to your use-case, use the id attribute of the mf object like this:

crc.with_output(mf.id)

 

Note that programmatically creating Flows can be quite challenging, so you should tread with caution and make sure to implement guardrails at sensitive places (e.g. checking if a managed folder already exists before creating it). 

Hope this helps!

Best,

Harizo

View solution in original post

2 Replies
HarizoR
Developer Advocate
Developer Advocate

Hi Daniele,

You almost had it in your Solution 1 🙂 The with_output() method of the DSSRecipeCreator takes as argument either the Dataset name or the Folder's identifier (see the documentation  for more details).

To apply this to your use-case, use the id attribute of the mf object like this:

crc.with_output(mf.id)

 

Note that programmatically creating Flows can be quite challenging, so you should tread with caution and make sure to implement guardrails at sensitive places (e.g. checking if a managed folder already exists before creating it). 

Hope this helps!

Best,

Harizo

dongari
Level 2
Author

SOLVED.
Thanks! It worked well.
I summarize the full working code for clarity:

crc = p.new_recipe(type='python', name="compute_B_from_A")
crc.with_input("A")
mf = p.create_managed_folder(name="B", connection_name='filesystem_folders')
crc.with_output(mf.id)
crc.with_script("print('hello world!')")
r = crc.create()
0 Kudos