Transfer or copy of files in a folder from one S3 bucket to another S3 bucket folder

Options
Nainish09
Nainish09 Registered Posts: 7

Hi Team,

I am curious to know, how can I import/transfer/copy a folder full of datasets in one of the s3 bucket lets say Bucket1 into my zone, so that i can work on the datasets.

Answers

  • JordanB
    JordanB Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 293 Dataiker
    Options

    Hi @Nainish09
    ,

    If I understand you correctly, it sounds like you want to move or share a folder to a new/different zone? You can right-click on the folder and choose between "share" or "move" folder to a flow zone. If you would like to share the folder between projects, you can do so by selecting the folder > Actions > Share and adding target projects.

    Please let us know if you have any further questions.

    Thanks!

    Jordan

  • qweenfo
    qweenfo Registered Posts: 3
    Options
    You can employ the AWS Command Line Interface (CLI) or an interface tool like AWS Management Console to copy a folder packed with datasets from a specific S3 bucket, such as Bucket1, to your local setup for analysis.
  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,708 Neuron
    Options

    Assuming you have two managed folders created in two s3 bucket connections, either the same s3 connection or a different s3 connection and that the destination s3 folder you have write permissions the easiest way to copy files is using the Merge Folder visual recipe. Select the input folder, then select Merge Folder visual recipe from the right pane, set the output folder using the "USE EXISTING" option at the bottom of the "New Merge Folder recipe and selected your output s3 folder. Then run the recipe and Dataiku will copy the files between the buckets.

    Screenshot 2024-06-06 225434.png

Setup Info
    Tags
      Help me…