Create a Recipe via API which outputs to a managed Folder
Dear Dataiku Community,
it currently seems impossible to create via API a Recipe which outputs to a managed folder instead of a Dataset. See the following code:
crc = p.new_recipe(type='python', name="compute_B_from_A") crc.with_input("A") # Solution 1 not working mf = p.create_managed_folder(name="B", connection_name='filesystem_folders') crc.with_output(mf) # Solution 2 not workin neither crc.with_new_output_dataset(name="B",connection='filesystem_folders') crc.with_script(crc3_code) r = crc.create() r.run(wait=True)
Actually, I can do it using the graphical interface:
Please let me know if there is a correct way to do it via API!
Thanks!
Danieel
Operating system used: UNIX
Operating system used: Unix
Best Answer
-
Hi Daniele,
You almost had it in your Solution 1
The with_output() method of the DSSRecipeCreator takes as argument either the Dataset name or the Folder's identifier (see the documentation for more details). To apply this to your use-case, use the id attribute of the mf object like this:
crc.with_output(mf.id)
Note that programmatically creating Flows can be quite challenging, so you should tread with caution and make sure to implement guardrails at sensitive places (e.g. checking if a managed folder already exists before creating it).
Hope this helps!
Best,
Harizo
Answers
-
SOLVED.
Thanks! It worked well.
I summarize the full working code for clarity:crc = p.new_recipe(type='python', name="compute_B_from_A") crc.with_input("A") mf = p.create_managed_folder(name="B", connection_name='filesystem_folders') crc.with_output(mf.id) crc.with_script("print('hello world!')") r = crc.create()