Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hi !!
I have two folders both connected to the same S3 bucket but not the same directory.
I want to copy a subset of first folder in the second folder with a recipe (python if possible).
I've already tried to do this with :
f = Folder_A.get_download_stream(filename) and Folder_B.upload_stream(file_copy, f)
But it's veeeery slow (5 min to copy a 20mo file)
Is there a better method to copy a file from bucket to bucket ?
Thank you !!
Hi,
The get_download_stream and upload_stream method imply data streaming in-and-out of S3 onto the DSS server.
You can achieve higher efficiency by using a cloud-specific API: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client
Cheers,
Alex
Hi,
The get_download_stream and upload_stream method imply data streaming in-and-out of S3 onto the DSS server.
You can achieve higher efficiency by using a cloud-specific API: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client
Cheers,
Alex