We're excited to announce that we're launching the second installment of Dataiku Product Days Register Now

How to add hyperparam optimization support to custom models?

Solved!
dadbuz
Level 1
How to add hyperparam optimization support to custom models?

Hi,

I've built a custom ML model following the documentation [ https://doc.dataiku.com/dss/latest/machine-learning/algorithms/in-memory-python.html#custom-models ] and would like to be able to leverage dataiku builtin cross-validation and hyperparameter search to find optimal parameter settings.

Thanks

 

0 Kudos
1 Solution
Marine
Dataiker
Dataiker

Hi @dadbuz , 

Today, the hyperparameter optimisation does not work for custom python models in auto ML. Grid search will work for plugin models. Converting a custom model into a plugin model is quite straightforward. For more information, you may have a look at this tutorial

Have you tried to package your model as a plugin? If yes, did you encounter any limitations? 

View solution in original post

0 Kudos
3 Replies
Marine
Dataiker
Dataiker

Hi @dadbuz ! 

During the training, you can select K-fold cross test both for default and custom ML models. Go to your Analysis > Design > Train / test set > 

 

You can set up grid search parameters for your custom models by adding fields into the algo.json file. Have a look at this documentation for more information : https://doc.dataiku.com/dss/latest/plugins/reference/prediction-algorithms.html#grid-search 

You can find an example of a custom model implementing managed grid search on Dataiku's github : https://github.com/dataiku/dss-plugin-model-lightgbm/blob/master/python-prediction-algos/model-light... 

 

0 Kudos
dadbuz
Level 1
Author

Hi @Marine,

Well, is there a way to invoke hypeparams optimisation without having to convert my custom model to a plugin?  

0 Kudos
Marine
Dataiker
Dataiker

Hi @dadbuz , 

Today, the hyperparameter optimisation does not work for custom python models in auto ML. Grid search will work for plugin models. Converting a custom model into a plugin model is quite straightforward. For more information, you may have a look at this tutorial

Have you tried to package your model as a plugin? If yes, did you encounter any limitations? 

View solution in original post

0 Kudos
A banner prompting to get Dataiku DSS