Timed File Update

Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner

I have a folder on my computer that contains a log file.

This log file is automatically updated and appended on to by a program on my machine.

Is there a way that after a certain amount of time, I can automate the workflow to pull the newly updated file?


Operating system used: Windows (VM)

Best Answers

  • Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 118 Dataiker
    Answer ✓

    Hi,

    Automating operations is the whole basis for the existence of Scenarios.

    The pulling of the updated file can be a step of this newly created Scenario. There are a wide range of steps to suit your needs, or you can code the entire Scenario in Python.

    The automation bit comes from having the possibility of specifying triggers that start Scenario runs. The particular trigger of interest in this case would be a time-based trigger one.

  • Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner
    Answer ✓

    So, when you make a folder, you can assign a file path that is on the machine

    Whenever you use a scenario to run, you can have it update the folder to ensure that it's fully updated!

    Thank you!

Answers

  • Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 42 Partner

    Is there a step that would probably work for looking at a folder on the host machine and pulling up the updated file?

Welcome!

It looks like you're new here. Sign in or register to get started.

Welcome!

It looks like you're new here. Sign in or register to get started.