Time Series Forecasting with Rolling Window

aguinier

Dear all,

Let's say that I want to make the forecast of the electricity demand. The goal is that everyday at 18:00 I forecast the electricity demand for all hour of the day after.

I want to train my algorithm on a rolling windows meaning: 6 month before to D-1 at 18:00. I also have at my disposal some weather forecast that give me an estimation of weather data for day D.

I am unable to find a way to perform this in the time series module.

I have the latest version of Dataiku.

Many thank for your help.

Best Regards,

---

Operating system used: docker

StanG

Hi,

You can perform just what you are explaining by using the Visual Time series forecasting capabilities of Dataiku. You'll need to set the forecast horizon to 24 hours, the time unit to 1h and to select as external feature the weather column to train models on your historical data.

Then, after deploying the model to the flow, you can use the Scoring recipe on a dataset that contains both your historical data and the weather forecast of the next day.

See this doc for more information: https://doc.dataiku.com/dss/latest/machine-learning/time-series-forecasting/index.html

Tell us if you encounter any issues while doing that.

aguinier

Hi,

Thanks for the answer.

The thing is that way it is not a rolling (or extending windows), I manually have to retrain,rescore, the model. Is there a way to automate that?

StanG

Once your model is deployed, you can retrain it easily on the updated data.

Everything in dataiku can be automated using scenarios. For instance, you could create a scenario to retrain and re-score the new data every day.

More details about scenarios here: https://doc.dataiku.com/dss/latest/scenarios/index.html

aguinier

Yes indeed it sounds like a good idea but it would be great to have such feature embeded in dataiku directly!

Unfortunately I work with the free version of dataiku so this feature is not available for me.