Discover this year's submissions to the Dataiku Frontrunner Awards and give kudos to your favorite use cases and success stories!READ MORE

How to add hyperparam optimization support to custom models?

Solved!
dadbuz
Level 2
How to add hyperparam optimization support to custom models?

Hi,

I've built a custom ML model following the documentation [ https://doc.dataiku.com/dss/latest/machine-learning/algorithms/in-memory-python.html#custom-models ] and would like to be able to leverage dataiku builtin cross-validation and hyperparameter search to find optimal parameter settings.

Thanks

 

0 Kudos
1 Solution
Marine
Dataiker
Dataiker

Hi @dadbuz , 

Today, the hyperparameter optimisation does not work for custom python models in auto ML. Grid search will work for plugin models. Converting a custom model into a plugin model is quite straightforward. For more information, you may have a look at this tutorial

Have you tried to package your model as a plugin? If yes, did you encounter any limitations? 

View solution in original post

0 Kudos
3 Replies
Marine
Dataiker
Dataiker

Hi @dadbuz ! 

During the training, you can select K-fold cross test both for default and custom ML models. Go to your Analysis > Design > Train / test set > 

 

You can set up grid search parameters for your custom models by adding fields into the algo.json file. Have a look at this documentation for more information : https://doc.dataiku.com/dss/latest/plugins/reference/prediction-algorithms.html#grid-search 

You can find an example of a custom model implementing managed grid search on Dataiku's github : https://github.com/dataiku/dss-plugin-model-lightgbm/blob/master/python-prediction-algos/model-light... 

 

0 Kudos
dadbuz
Level 2
Author

Hi @Marine,

Well, is there a way to invoke hypeparams optimisation without having to convert my custom model to a plugin?  

0 Kudos
Marine
Dataiker
Dataiker

Hi @dadbuz , 

Today, the hyperparameter optimisation does not work for custom python models in auto ML. Grid search will work for plugin models. Converting a custom model into a plugin model is quite straightforward. For more information, you may have a look at this tutorial

Have you tried to package your model as a plugin? If yes, did you encounter any limitations? 

0 Kudos