Do not update or write intermediate dataset

dhyadav79
Level 1
Do not update or write intermediate dataset

Hi,

I have flow where I do not update/write intermediate dataset.

Could you please help, how can I achieve this.

 

0 Kudos
1 Reply
AlexT
Dataiker

Hi @dhyadav79 ,

If you are using  SQL Pipeline or Spark Pipelines avoid building intermediate datasets. 
by seting the dataset “Virtualizable in build” See:

 https://doc.dataiku.com/dss/latest/sql/pipelines/sql_pipelines.html#configuring-behavior-for-interme... 


If you are not using pipelines but you simply want to protect this dataset, then set it to "write-protected"  Advanced tab -  Rebuild behavior 

Screenshot 2023-08-09 at 10.17.24 PM.png

0 Kudos