Threshold with live data

VincentSider
VincentSider Partner, Registered Posts: 7 Partner

Hello,

When I use Random Forest Classification to predict a probability and apply a threshold to it, is that threshold "frozen" when I then apply real data to the model?

For example, I have made a business decision, I have favoured precision as a metric based on a certain threshold and I want to ensure that the real data uses that threshold.

Many thanks for your help

Vincent

Answers

  • JeremieP
    JeremieP Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner Posts: 7 Dataiker

    Hi Vincent,

    When you will use the Score or the Predict recipes to predict a dataset with your model, you will be able to use the threshold that you want in the Settings tabs of the recipe as shown in this screenshot :

    score_test_prepared_-_Recipe___Dataiku.png

    Hope this helps.

    Jérémie

  • VincentSider
    VincentSider Partner, Registered Posts: 7 Partner

    thank you @JeremieP

    So when I publish the model and use live data, it will respect the threshold I selected (unless someone changes it), right? What I'm trying to do is prevent the threshold from being changed after I've made a decision.

  • JeremieP
    JeremieP Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner Posts: 7 Dataiker

    Yes you're right. The threshold will only change if someone changes it manually or if you retrain your model and deploy a new version of it with a new optimal threshold

Setup Info
    Tags
      Help me…