The Dataiku Frontrunner Awards have just launched to recognize your achievements! SUBMIT YOUR ENTRY

Accessing object storage (S3) in DellEMC Powerscale storage

ismail
Level 2
Accessing object storage (S3) in DellEMC Powerscale storage

Hi, 

May I know if DSS is able to connect to a DellEMC Powerscale with S3 protocol hosting object storage? I do know that S3 to AWS is possible but what about local enterprise storage systems with S3 protocol enabled? Thanks!

0 Kudos
4 Replies
TheophileT
Dataiker
Dataiker

Hi,

I don't know DellEMC powerscale, but if you can use virtual hosted-style access to your bucket (like https://bucket-name.s3.Region.amazonaws.com/key name) that should work without any problem.

0 Kudos
wilfred4510
Level 1
Level 1

Dell EMC powerscale is a physical Scale out NAS storage solution that provides multiple access options for data stored within it which includes S3 protocol access. It supports multiple simultaneous access protocols. We are able to connect to the S3 bucket on powerscale using S3 browser when we select S3 compatible storage. Does Dataiku support S3 access to object storage on Physical Storage systems that provide S3 access support? Please forgive me if I misunderstood your response. I believe virtual hosted access is for multiple access to a single S3 bucket on AWS which is not what we are trying to achieve.

0 Kudos
TheophileT
Dataiker
Dataiker

Hi @wilfred4510,

Sorry if I wasn't clear. Dataiku supports S3 access from any S3 compatible storage.

But there are two kinds of url format.

  • Path-style URLs like
https://s3.Region.amazonaws.com/bucket-name/key name
  • and virtual-hosted-style URLs like
https://bucket-name.s3.Region.amazonaws.com/key name

 

Because of how things are done in the UI, Dataiku uses virtual-hosted-style URLs (That will be configurable in future DSS version).

In the powerscale doc I see:

https://www.delltechnologies.com/resources/en-us/asset/white-papers/products/storage/h18293-dell-emc... wrote:
Starting with OneFS 9.0, PowerScale OneFS supports the Amazon S3 protocol with OneFS S3, an objectstorage interface that is compatible with the Amazon S3 API.

OneFS supports both path-style requests and virtual hosted-style requests.

So you shouldn't have any problem.

 

0 Kudos
DrissiReda
Level 3

I see that in 9.0 release notes, it is supported:

 

" S3: It is now possible to force “path-style” rather than “virtualhost-style” S3 access. This is mainly useful for “S3-compatible” storages."

 

How to configure it? I can't find anything in the UI.

A banner prompting to get Dataiku DSS
Public