uploading file to S3 bucket
I have a file and path on S3 bucket is fixed. So, when I upload file to S3 bucket, it automatically appends extra folder to the existing path. This limits it from further auto processing. How can I limit the append of extra path?
Moreover, it uploads file in S3 as out.csv.gz How can I upload file.withy desired file name?
While exporting dataset via export recipe, it creates a folder with random recipe name. How can I restrict from creating the folder?
Answers
-
Hi,
Unfortunately, I am unable to reproduce the situation you are describing.
Could you please detail the different steps by adding some screenshots of your flow and recipe settings?
Cheers,
Alex
-
piyushk Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Adv Designer, Registered Posts: 55 ✭✭✭✭✭
S3 Connection:
Content of Path prefix: dataiku_test_db/dataiku_test/snapshot/2020/04/10/
------------------------------------------------------------------------------------------------------
I am exporting a dataset to a fixed path on S3 bucket. So, when I upload file to S3 bucket, DSS automatically appends extra folder to the existing fixed path. This limits it from further auto processing. How can I limit the append of extra path?
Extra path here added is Orders_Filtered_Demo.
-------------------------------------------------------------------------------------------------------------------
DSS uploads file in S3 as out.csv.gz How can I upload file with desired file name like FileName.csv?
---------------------------------------------------------------------------------------------------------------------------------------
While exporting dataset via export recipe, it creates a folder with random recipe name. How can I restrict from creating the folder with random name?
-
Hi,
Thanks for the precisions, it helps. I suggest using the "Sync" recipe instead of the "Export recipe".
After creating the sync recipe, you'll be able to go in the output dataset and edit the settings you are referring to:
- Path in bucket
File format / extension
File name
Hope it helps,
Alex