Discover all of the brand-new features and improvements to existing capabilities in the Dataiku 11.3 updateLET'S GO

Timed File Update

Solved!
mahmoud_shihab
Level 3
Timed File Update

I have a folder on my computer that contains a log file.

This log file is automatically updated and appended on to by a program on my machine.

Is there a way that after a certain amount of time, I can automate the workflow to pull the newly updated file?


Operating system used: Windows (VM)

0 Kudos
2 Solutions
MiguelangelC
Dataiker

Hi,

Automating operations is the whole basis for the existence of Scenarios.

The pulling of the updated file can be a step of this newly created Scenario. There are a wide range of steps to suit your needs, or you can code the entire Scenario in Python.

The automation bit comes from having the possibility of specifying triggers that start Scenario runs. The particular trigger of interest in this case would be a time-based trigger one.

View solution in original post

mahmoud_shihab
Level 3
Author

So, when you make a folder, you can assign a file path that is on the machine

Whenever you use a scenario to run, you can have it update the folder to ensure that it's fully updated!

Thank you!

View solution in original post

3 Replies
MiguelangelC
Dataiker

Hi,

Automating operations is the whole basis for the existence of Scenarios.

The pulling of the updated file can be a step of this newly created Scenario. There are a wide range of steps to suit your needs, or you can code the entire Scenario in Python.

The automation bit comes from having the possibility of specifying triggers that start Scenario runs. The particular trigger of interest in this case would be a time-based trigger one.

mahmoud_shihab
Level 3
Author

Is there a step that would probably work for looking at a folder on the host machine and pulling up the updated file?

0 Kudos
mahmoud_shihab
Level 3
Author

So, when you make a folder, you can assign a file path that is on the machine

Whenever you use a scenario to run, you can have it update the folder to ensure that it's fully updated!

Thank you!