Automating Flow

m_schneids
Level 1
Automating Flow

Hi 

I have recently finished creating a flow that ingests data from a database, then it is run through an r recipe and then a sync recipe to upload to an S3 bucket. 

I am now looking to automate this process. Is it possible to set up a scenario that runs through the flow and then upload the resulting dataset to a S3 bucket? 

0 Kudos
3 Replies
Manuel
Dataiker Alumni

Hi,

Yes, the functionality to automate a flow is called scenarios:

I hope this helps.

 

0 Kudos
m_schneids
Level 1
Author

Hi, 

Thanks for sharing these resources. I have set up a scenario and a trigger that runs every 5 minutes to build my datasets and run my recipes. I get a message saying the scenario was successful but my data is not updated in S3? Do you know why this would be? Because my datasets did not change in the 5 minutes? I set the trigger to go every 5 minutes to verify that it was working. 

Thanks 

0 Kudos
Manuel
Dataiker Alumni

Hi,

Yes, if the input data does not change and the flow is the same, then the output should be the same.

If you want to be certain, clear the data from your output dataset and wait for the scenario to run again.

I hope this helps.

 

0 Kudos