Discover all of the brand-new features and improvements to existing capabilities in the Dataiku 11.3 updateLET'S GO

Writing to S3 Bucket with 'Object Lock'

benmoss
Level 2
Writing to S3 Bucket with 'Object Lock'

Hi all,

We have an S3 bucket with 'Object Lock' enabled for the purpose of regulatory requirements, this means that once a file is written, that original version is stored for a set-period of time and cannot be removed.

The Object Lock is enabled for any additional file added to the bucket.

However, we have bumped into an issue when trying to write data into this bucket, specifically we recieve an error which indicates that the 'Content-MD5 OR x-amz-checksum- HTTP header is required for Put Part requests with Object Lock parameters'.

This is also outlined in AWS's documentation for object lock objects: https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html

Has anyone any experience with writing data into an object lock bucket that can advise on how they worked around this issue?


Operating system used: Windows

0 Kudos
1 Reply
AlexT
Dataiker

Hi @benmoss ,

Activating "Object Lock" mode will require calculating MD5 for each call to S3.  DSS does not support this option.

You can submit this to https://community.dataiku.com/t5/Product-Ideas/idb-p/Product_Ideas for future consideration. 

Thanks

 

0 Kudos