Dataiku Webapp using Datasets not getting Refreshed
I have created a webapp using Dash in Dataiku which connects to few datasets within the project and also helps the user to run scenarios via the app. The issue i am currently facing is that whenever any changes are done to the datasets in dataiku the same is not reflected in the webapp, I have tried using page refresh and also interval, but the data does not change unless i make some changes to the code and save the webapp again.
Answers
-
The behaviour you describe indicates that the webapp's python backend loads the dataset upon starting and then serves this data. So unless you stop and restart the webapp backend, the data served remains the state of the data at the time of the backend startup.
If you want the webapp to react to changes in the data you need:
- a way to detect that the dataset's data has changed. For example you can hash the output of https://doc.dataiku.com/dss/latest/python-api/datasets-reference.html#dataiku.Dataset.get_files_info
- a way to reload the data while the webapp backend is running. Either a background thread in the python backend, or a hook on the Dash callbacks to check for changes and reload if necessary, or an explicit reload button in the webapp...
-
I did try reloading the page manually as well as placing a reload button and using that to refresh the page, but the updated data is still not read into the webapp, But yes let me try to use the get_files_info as suggested by you to check for any changes in the dataset, i will update here if it works.
