leverage automatic fast-write from s3 to snowflake using private link
Hello Dataiku team,
We need to write data into snowflake from dataiku via AWS S3 in the meanwhile ensuring all traffic goes with private link instead of exposing to the public internet.
How to set that up?
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,225 DataikerHi,
I guess this guide would apply DSS would not be involved here it would be configured on AWS/Snowflake side to make sure the stage defined goes trough the private link.
https://aws.amazon.com/blogs/apn/using-aws-privatelink-for-amazon-s3-for-private-connectivity-between-snowflake-and-amazon-s3/
For DSS to to snowflake on private link you need to ensure the hostname used the privatelink one :
https://community.snowflake.com/s/article/AWS-PrivateLink-and-Snowflake-detailed-troubleshooting-Guide
If you use a proxy you may need to set no_proxy env variables accordingly to not send snowflake traffic through the proxy.
Thanks,