Submit your inspiring success story or innovative use case to the 2022 Dataiku Frontrunner Awards! ENTER YOUR SUBMISSION

Optimize therehsold options in dataiku

samrudhak
Level 1
Optimize therehsold options in dataiku

Dataiku provides many options to evalate model in the section"Optimize model hyperparameters for" 

However there are only 3 options in the section below it which is "Optimize threshold for"

Why are others parameters (like AUC) are removed for the list?

0 Kudos
1 Reply
Clément_Stenac
Dataiker
Dataiker

Hi,

Optimizing threshold can only use metrics that actually depend on the threshold. AUC and log loss are "global" metrics that look at all probabilities, so it would not make sense to use them to optimize the threshold (they are fully independent from it).

Precision and Recall are also excluded because "optimizing for precision" would always yield 1 as threshold, and "optimizing for recall" would always yield 0. You need metrics that represent a balance between these, which is why F1 or cost-matrix are almost always what you want.

0 Kudos

Labels

?
Labels (1)
A banner prompting to get Dataiku