Community Conundrum 28: News Engagement is live! Read More

copy files from bucket to bucket

Level 1
copy files from bucket to bucket

Hi !!

I have two folders both connected to the same S3 bucket but not the same directory.

I want to copy a subset of first folder in the second folder with a recipe (python if possible). 

I've already tried to do this with :

f = Folder_A.get_download_stream(filename) and Folder_B.upload_stream(file_copy, f)

But it's veeeery slow (5 min to copy a 20mo file)

Is there a better  method to copy a file from bucket to bucket ?

Thank you !!

1 Reply


The get_download_stream and upload_stream method imply data streaming in-and-out of S3 onto the DSS server.

You can achieve higher efficiency by using a cloud-specific API:



Labels (1)
A banner prompting to get Dataiku DSS