Automate selecting champion model and challenger model

devapavan

I have 3 models under model deployment out of which one is active version and other two are inactive. Is there a way to automatically select these models over the time whichever is performing better? For example: Model that is active today might not be giving the accuracy after a month whereas the model which is inactive today can give good accuracy after a month. Is there a way to select automatically my champion and challenger models?

JordanB

Hi @devapavan
,

You can automate the task of activating the best-performing model using the following article to guide you: https://blog.dataiku.com/mlops-champion-challenger-model-evaluation

When it comes to deploying it, you can add a step in your scenario to update your current api deployment. https://community.dataiku.com/t5/Dataiku-Inspiration/Automate-deployment-on-API-node/ba-p/22169

https://knowledge.dataiku.com/latest/mlops-o16n/batch-deployment/tutorial-update-project-deployment.html

Thanks!

Jordan