Discover this year's submissions to the Dataiku Frontrunner Awards and give kudos to your favorite use cases and success stories!READ MORE

Loading a Spark Dataframe to snowflake , python3.8 upgrade

verleger
Level 1
Level 1
Loading a Spark Dataframe to snowflake , python3.8 upgrade

 How do I write a spark Dataframe to Snowflake table  from Jupyter notebook in Dataiku ? 

so much document, very little examples !

Thanks

0 Kudos
1 Reply
Clément_Stenac
Dataiker
Dataiker

Hi,

Assuming that "myoutput" is the name of the Snowflake dataset in which you want to write, and that "df" is the Spark dataframe, you would write:

import dataiku
from dataiku import spark as dkuspark

myoutput = dataiku.Dataset("myoutput")
dkuspark.write_with_schema(out, df)

 

https://doc.dataiku.com/dss/latest/code_recipes/pyspark.html

Best,

0 Kudos