Timed File Update

mahmoud_shihab
mahmoud_shihab Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner

I have a folder on my computer that contains a log file.

This log file is automatically updated and appended on to by a program on my machine.

Is there a way that after a certain amount of time, I can automate the workflow to pull the newly updated file?


Operating system used: Windows (VM)

Best Answers

  • Miguel Angel
    Miguel Angel Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 118 Dataiker
    Answer ✓

    Hi,

    Automating operations is the whole basis for the existence of Scenarios.

    The pulling of the updated file can be a step of this newly created Scenario. There are a wide range of steps to suit your needs, or you can code the entire Scenario in Python.

    The automation bit comes from having the possibility of specifying triggers that start Scenario runs. The particular trigger of interest in this case would be a time-based trigger one.

  • mahmoud_shihab
    mahmoud_shihab Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner
    Answer ✓

    So, when you make a folder, you can assign a file path that is on the machine

    Whenever you use a scenario to run, you can have it update the folder to ensure that it's fully updated!

    Thank you!

Answers

  • mahmoud_shihab
    mahmoud_shihab Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner

    Is there a step that would probably work for looking at a folder on the host machine and pulling up the updated file?

Setup Info
    Tags
      Help me…