Accessing object storage (S3) in DellEMC Powerscale storage
Hi,
May I know if DSS is able to connect to a DellEMC Powerscale with S3 protocol hosting object storage? I do know that S3 to AWS is possible but what about local enterprise storage systems with S3 protocol enabled? Thanks!
Answers
-
Hi,
I don't know DellEMC powerscale, but if you can use virtual hosted-style access to your bucket (like https://bucket-name.s3.Region.amazonaws.com/key name) that should work without any problem.
-
Dell EMC powerscale is a physical Scale out NAS storage solution that provides multiple access options for data stored within it which includes S3 protocol access. It supports multiple simultaneous access protocols. We are able to connect to the S3 bucket on powerscale using S3 browser when we select S3 compatible storage. Does Dataiku support S3 access to object storage on Physical Storage systems that provide S3 access support? Please forgive me if I misunderstood your response. I believe virtual hosted access is for multiple access to a single S3 bucket on AWS which is not what we are trying to achieve.
-
Hi @wilfred4510
,Sorry if I wasn't clear. Dataiku supports S3 access from any S3 compatible storage.
But there are two kinds of url format.
- Path-style URLs like
https://s3.Region.amazonaws.com/bucket-name/key name
- and virtual-hosted-style URLs like
https://bucket-name.s3.Region.amazonaws.com/key name
Because of how things are done in the UI, Dataiku uses virtual-hosted-style URLs (That will be configurable in future DSS version).
In the powerscale doc I see:
https://www.delltechnologies.com/resources/en-us/asset/white-papers/products/storage/h18293-dell-emc-powerscale-onefs-s3-api-guide.pdf wrote:
Starting with OneFS 9.0, PowerScale OneFS supports the Amazon S3 protocol with OneFS S3, an objectstorage interface that is compatible with the Amazon S3 API.OneFS supports both path-style requests and virtual hosted-style requests.
So you shouldn't have any problem.
-
I see that in 9.0 release notes, it is supported:
" S3: It is now possible to force “path-style” rather than “virtualhost-style” S3 access. This is mainly useful for “S3-compatible” storages."
How to configure it? I can't find anything in the UI.
