Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
How do I write a spark Dataframe to Snowflake table from Jupyter notebook in Dataiku ?
so much document, very little examples !
Thanks
Hi,
Assuming that "myoutput" is the name of the Snowflake dataset in which you want to write, and that "df" is the Spark dataframe, you would write:
import dataiku
from dataiku import spark as dkuspark
myoutput = dataiku.Dataset("myoutput")
dkuspark.write_with_schema(out, df)
https://doc.dataiku.com/dss/latest/code_recipes/pyspark.html
Best,