Loading a Spark Dataframe to snowflake , python3.8 upgrade

verleger · August 2022

How do I write a spark Dataframe to Snowflake table from Jupyter notebook in Dataiku ?

so much document, very little examples !

Thanks

Clément_Stenac · August 2022

Hi,

Assuming that "myoutput" is the name of the Snowflake dataset in which you want to write, and that "df" is the Spark dataframe, you would write:

import dataiku
from dataiku import spark as dkuspark

myoutput = dataiku.Dataset("myoutput")
dkuspark.write_with_schema(out, df)

Best,

Answers