Sync a dataset to S3 with headers, no compression and custom name

Romain_NIO
Romain_NIO Registered Posts: 12 ✭✭✭✭
Hi,

I create a workflow to normalize a dataset imported from AWS S3 bucket.I added a "prepare recipe" and it works fine.

I want to export this dataset to S3, with a custom name, header, and no-compression (this file will be processed by an external tool) but I don't see any options to configure the export while using "sync recipe". The filename is always "0.csv.gz" without header and with compression activated.

Any ideas ?

Answers

  • Thomas
    Thomas Dataiker Alumni Posts: 19 ✭✭✭✭✭

    Hi Romain,

    To add the headers and remove compression, you just need to open your S3 dataset settings, and resp. click the "Parse next line as column headers" option, and set "File Compression" to None:

    Not sure if this is possible to rename the file, I'll let others jump in.

    Thanks

  • Romain_NIO
    Romain_NIO Registered Posts: 12 ✭✭✭✭
    Hi Thomas,

    Thanks for your answer. For the name of the file, I'm interested by having the answer if you have it.
Setup Info
    Tags
      Help me…