Eval vs score recipe

Options
NR
NR Registered Posts: 14

I'm using Time series Forecasting in Dataiku. after training model, I use eval recipe to evaluate validation set and score recipe to make prediction for new unseen data.

Typically, If I use the scoring recipe for the validation set, it should give the same result as the eval recipe but I'm getting different predictions.

To do this experimentation, I filtered in a new data a range from the validation data then applied the score recipe then I started to compare per date, the score recipe forecast with the eval recipe forecast previously done and found gap. any idea about the difference explanation ?

Typically if the model isn't retrained, I should have the same result for the same data either with eval or score recipe.

I did it for the time series sample project as an example :

https://knowledge.dataiku.com/latest/ml-analytics/time-series/tutorial-forecasting-visual-ml.html#explore-the-evaluation-results

and I'm sharing the here the results

Tagged:

Best Answer

Setup Info
    Tags
      Help me…