Web app does not update

aseth1 Registered Posts: 1 ✭✭✭✭
I have a scenario running that pulls data daily to run a model. The resulting data table updates everyday. I am then pulling the data in a Webapp to plot it using a javascript library. For some reason the plots in the webapp do not update daily. I have to restart the Python backend to refresh the data. Any solutions?


  • jereze
    jereze Alpha Tester, Dataiker Alumni Posts: 190 ✭✭✭✭✭✭✭✭
    edited July 17


    The python method to get the dataframe (dataiku.Dataset(name, project).get_dataframe()) could be executed at query time instead of at webapp init. This way, the data will be updated at each request.

    However, that might slow down your queries. You should consider a caching system. For xemaple:

    cached_dfs = {}
    def get_cached_dataframe(name, project):
    global cached_dfs

    if name not in cached_dfs.keys() or datetime.datetime.now() - cached_dfs[name][0] > datetime.timedelta(minutes=30):
    cached_dfs[name] = (datetime.datetime.now(), dataiku.Dataset(name, project).get_dataframe())

    return cached_dfs[name][1]

    # then in your code, when you need to get a dataframe object:
    my_df = get_cached_dataframe("dataset_name", "project_key")

Setup Info
      Help me…