Updating a sharepoint list or api generated dataset via trigger

jacobcalka

Hi,

I'm working with the rest api and sharepoint online plugins to create datasets in dataiku (two separate sets). I want to create a scenario that runs every once in a while and pulls new data that is populating the sharepoint and api location into the current dataset in dataiku. Right now, the automation runs, but the dataset does not repopulate. However, when there is new data in the sharepoint list or api location, and I go into datasets and click "update sample," the dataset is populated. This gives me hope, but I really just need a step that will essentially press that button without me having to do it. I've attached my current trigger settings and steps. Don't know if this is possible due to the fact I'm leveraging plug ins... Any help would be appreciated. Thank you!

jacobcalka

The issue is solved by setting the build step to the most downstream dataset. If you just want to update a dataset, adding a sync to it then setting the new dataset as the one to build will do the trick.

Turribeach

In the Build Mode in the first step can you change it to Force Rebuild? Also be aware that the Sample is not the same thing as the whole dataset. Both of these can refresh at different times depending on what you do.