Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Write in existing and used folder
Added on March 3, 2021 1:58PM
Likes: 0
Replies: 3
Mingo
Registered Posts: 7 ✭✭✭
Hello,
I would like to export all my files created by dataiku ( csv or other ) in a single folder.
But when I choose to output a recipe in an already existing and already used folder, it doesn't work.
I would prefer not to create a folder per output, is it feasible?
Thanks in advanced
Answers
-
Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 319 DataikerHi @Mingo
,Can you explain which type of "output" you are selecting when you attempt to select an output folder that already exists? Are you referring to using an existing "managed folder" when you select your output?
If so, I think one possible alternative is to configure your output folder at the connection level instead, and use a single filesystem "connection" so that all recipe outputs are to this connection, which is really just a folder on your server/local system. I'll highlight an example of exporting datasets from different projects to the same connection, which all export to the same overall folder.
I have a connection set up on my local system that will write to the folder "my-single-folder":
To illustrate the output, I created sync recipes in two different projects, that both will write to this folder.
And in another project, just select the same connection:
These will now all land in the same "folder" on my system. If you also need a "managed folder" that points to the same location as this directory, you could then create a managed folder that points to this connection path.
Let me know if this doesn't address your use case or if you have any questions.
Thanks,
Sarina
-
Hello Sarina and thanks for your answer.
I'm using the Export to Folder procedure and I'm creating a qvx file.
So I have to create one object 'folder' , one for each export
At the end I agree, they all end at the same folder on the system but I :
- I have to create one folder for each output.
- If I want to change the path of my outputs I have to change it twice (or more)
In fact I would like to do this :
And I'm just wondering if it's technically feasible.
Thanks
-
Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 319 Dataiker
Hi @Mingo
,Thank you for clarifying your use case and posting the screenshots. Indeed, it is not possible to use the same output folder in multiple "Export to folder" recipes.
Though it would still require all the same initial setup, one option might be to use the "Merge folders" recipe based on all your input folders to consolidate them into 1 final folder:
Thanks,
Sarina