Creating a managed folder in recipe

Highlighted
tomas
Level 4
Creating a managed folder in recipe
Jump to solution

Hi,



 I try to create a managed folder in pyspark recipe, the folder is created, and then attached as an output to the recipe, but a few lines later when I try to access it it fails:



 Managed folder LWVG21ww cannot be used : declare it as input or output of your recipe



 



Maybe the DSS is having the old version of recipe outputs in cache and does not realize that it has been changed during the execution. Is there a way to to change it?



Thanks

0 Kudos
1 Solution

Accepted Solutions
Clément_Stenac Dataiker
Dataiker
Re: Creating a managed folder in recipe
Jump to solution
Hi,

It is not possible to dynamically modify Flow structure of a recipe while this recipe is running.

View solution in original post

0 Kudos
2 Replies
Clément_Stenac Dataiker
Dataiker
Re: Creating a managed folder in recipe
Jump to solution
Hi,

It is not possible to dynamically modify Flow structure of a recipe while this recipe is running.

View solution in original post

0 Kudos
Graham_E
Level 2
Re: Creating a managed folder in recipe
Jump to solution

Not sure what you are trying to accomplish but I made a recipe that dynamically creates sub-folders of a managed folder as needed. I use the OS module to make the directories.




import dataiku
import os

# OUTPUT_FOLDER is the name of the managed folder
folder_handle = dataiku.Folder(OUTPUT_FOLDER)

# NEW_FOLDER_NAME is the name of the folder you want to create
os.mkdir(folder_handle.get_path() + "/" + NEW_FOLDER_NAME)


From there you can save files to the new folder like you normally would. The OUTPUT_FOLDER needs to be created before running this code. Use the name of a managed folder that you create in the flow of your project.

0 Kudos
Labels (4)