Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Spark delta lake write support and structured streaming to delta tables
Added on May 18, 2022 6:56PM
Likes: 1
Replies: 0
nod
Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 2 ✭✭✭
Delta lake is an open format thats already supported on open source spark, we would like to be able to write delta tables directly from dataiku with spark recipes.
This would enable more ETL use cases, and with spark structured streaming with delta tables better management of streaming data.