Eval vs score recipe

Solved!
NR
Level 3
Eval vs score recipe

I'm using Time series Forecasting in Dataiku. after training model, I use eval recipe to evaluate validation set and score recipe to make prediction for new unseen data.

Typically, If I use the scoring recipe for the validation set, it should give the same result as the eval recipe but I'm getting different predictions.

To do this experimentation, I filtered in a new data a range from the validation data then applied the score recipe then I started to compare per date, the score recipe forecast with the eval recipe forecast previously done and found gap.   any idea about the difference explanation ?

Typically if the model isn't retrained, I should have the same result for the same data either with eval or score recipe. 

I did it for the time series sample project as an example :

https://knowledge.dataiku.com/latest/ml-analytics/time-series/tutorial-forecasting-visual-ml.html#ex...

and I'm sharing the here the results

0 Kudos
1 Solution
JordanB
Dataiker

Hi @NR,

A similar question was asked in the following community post: https://community.dataiku.com/t5/Using-Dataiku/Predict-recipe-vs-Evaluate-recipe/td-p/17055

As mentioned in the response, please try checking the box "Force original backend," rerun, and check if it yields the same result as the Evaluate recipe.

Thanks!

Jordan

View solution in original post

1 Reply
JordanB
Dataiker

Hi @NR,

A similar question was asked in the following community post: https://community.dataiku.com/t5/Using-Dataiku/Predict-recipe-vs-Evaluate-recipe/td-p/17055

As mentioned in the response, please try checking the box "Force original backend," rerun, and check if it yields the same result as the Evaluate recipe.

Thanks!

Jordan