Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hello, I have a problem with Dataiku's predictions, I don't know how they work. I am creating a model to get sales predictions. So far no problem, but when I try to reproduce the model, using the same variables, the same algorithm, the same file structure, the result is different from the one I had initially .The Rยฒ and the mean squared error changes also, especially with Random forest and XGboost. I don't understand why, is this normal?
Hi,
There are lots of things that could play here.
The most important one by far is that Dataiku preprocesses your data. This includes numerous things, but on "normal" data, the two most important ones are dummifying categorical values, and standardizing numericals.
In order to get as close as possible to the results of Visual ML using external code, you'd need to disable as much as possible of it, by:
Another thing that may be different is train/test split. In order to get more reproducible results, you'd want to pass explicit train and test sets to both Visual ML and your code
Then there is hyperparameter search. The easiest here would be to disable it entirely.
Hi,
There are lots of things that could play here.
The most important one by far is that Dataiku preprocesses your data. This includes numerous things, but on "normal" data, the two most important ones are dummifying categorical values, and standardizing numericals.
In order to get as close as possible to the results of Visual ML using external code, you'd need to disable as much as possible of it, by:
Another thing that may be different is train/test split. In order to get more reproducible results, you'd want to pass explicit train and test sets to both Visual ML and your code
Then there is hyperparameter search. The easiest here would be to disable it entirely.