Refresh/Reload Dataset

Options
Xundu
Xundu Registered Posts: 5 ✭✭✭

Can't auto reload the inputdataset from MySQL connection when the database's content changed?


Operating system used: Windows 10

Best Answer

  • Manuel
    Manuel Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 193 ✭✭✭✭✭✭✭
    Answer ✓
    Options

    See this page in the documentation.

    For SQL databases, changes are not automatically detected. You will need to implement an SQL query to trigger your scenario

    I hope this helps

Answers

  • Manuel
    Manuel Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 193 ✭✭✭✭✭✭✭
    Options

    Hi,

    When a recipe is executed, it always works the current state of the dataset.

    If you are referring to seeing a dataset in the User Interface, what you see is only a sample of the dataset. If you need it to be refreshed you can do Configure Sample > Save & Refresh.

    I hope this helps.

  • Xundu
    Xundu Registered Posts: 5 ✭✭✭
    Options

    Thank for your reply. I have Configure Sample > Save & Refresh, but it will not auto reload again when the MySQL's content changed.

    config_20220506212502.png

  • Manuel
    Manuel Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 193 ✭✭✭✭✭✭✭
    Options

    Yes, that is the behaviour. Refreshing the sample is a manual action.

    Why is it important for you for the sample to be refreshed automatically?

    The samples are there to help you design the pipeline of recipes. When you actually run a recipe, either manually or automatically in a scenario, it always works on the current dataset.

  • Xundu
    Xundu Registered Posts: 5 ✭✭✭
    Options

    I create a scenarios with a triggers in Dataset modified so that I can call the scenarios when I change the database's content from C# program.

  • Manuel
    Manuel Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 193 ✭✭✭✭✭✭✭
    Options

    ok.

    That is unrelated to the samples you see in the UI.

    Is the scenario not triggering with dataset changes? Is that the real issue?

    Thanks

  • Xundu
    Xundu Registered Posts: 5 ✭✭✭
    Options

    Thank you very much, I will try the document's method later.

  • Xundu
    Xundu Registered Posts: 5 ✭✭✭
    Options

    After created a trigger on SQL query change, the scenarios could be called when the database's content changed. But I need reload the InputDataSet from database automatically other than click "SAVE AND REFRESH SAMPLE", because I will change the database's content from C# program for many times when I need. Can you provide some suggestions or methods?

Setup Info
    Tags
      Help me…