Try your hand at analyzing royal sentiment in Dataiku DSS! Learn more

Use a managed folder as output of several recipes ?

Level 3
Use a managed folder as output of several recipes ?

Hello,

I have a managed folder containing several result files from an external API. I would like to call this API several times in different recipes but store the results in the same folder each time. Is it doable ?

Thx for your help

2 Replies
Dataiker
Dataiker

Hi @MGad 

In the flow indeed DSS won't allow you to put as an output an existing folder where this folder is already the output of another recipe. We do this because it could lead to conflicts and confusion. 

However, this can still be achieved. If you already have a folder on your flow, get the folder id from the URL path when opening the folder. The path will be something like

 

DSS_HOST/projects/projectKey/managedfolder/folderId/view/

 

 

Then, in your flow click on +Dataset > Folder to create a new folder. You will need to match the connection to that of the folder we will mirror. Then in Settings > Storage > Path replace the ${odbId}  part with the folderId from above.

From a DSS perspective, these will be two distinct folders and you will be able to have them as outputs of different recipes. But if you look at the contents you'll see they're the same as they point to the same location.

Keep in mind what I mentioned at the beginning of the post re keeping your design clear as otherwise, it might lead to issues in your project later on.

I hope this helps!

Level 3
Author

Alright, thank you for your clear answer 🙂