Intermediate storage option

Renga3037
Level 1
Intermediate storage option

Hi All

I have two task(which was done in single query in oracle) but in DSS available in different recipe.

Once first recipe completed I had to store that in oracle but I dont want to store every recipe output in oracle and read again for second task to complete. 

Is there any workaround to make this possible ?

Any intermediate store in DSS (In memory or Disk) by chance ?

Thanks in advance !! 

 

0 Kudos
1 Reply
fchataigner2
Dataiker

Hi,

DSS materializes recipe outputs, always. You can then explore the data and check the recipe's conformance to what you expected.

If you need to squeeze out intermediate dataset to save space, you can convert groups of virual recipes running on the SQL db to SQL recipes (using the 'convert to sql recipe' button in their output section), then assemble the generated SQL queries in a single SQL recipe, using the tools that the DB offers (temporary tables or views, WITH clauses...). Alternatively, if using an experimental feature is ok, you can activate the SQL pipelines in your project's settings and flag the datasets you don't want materialized as virtualizable.