Loading a Spark Dataframe to snowflake , python3.8 upgrade

verleger

How do I write a spark Dataframe to Snowflake table from Jupyter notebook in Dataiku ?

so much document, very little examples !

Thanks

Clément_Stenac

Hi,

Assuming that "myoutput" is the name of the Snowflake dataset in which you want to write, and that "df" is the Spark dataframe, you would write:

import dataiku
from dataiku import spark as dkuspark

myoutput = dataiku.Dataset("myoutput")
dkuspark.write_with_schema(out, df)

https://doc.dataiku.com/dss/latest/code_recipes/pyspark.html

Best,