Add "work rate" line to ML Decision Chart

Options
Marlan
Marlan Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Dataiku Frontrunner Awards 2021 Participant, Neuron 2023 Posts: 316 Neuron

The Decision Chart in the Performance section of the screen you see after training a model in Visual ML is a really nice tool. (As are all of the visuals in the Performance section.)

However, it'd be even more useful if it included a "work rate" line. This would show what percent of the cases would be considered a "positive" at each Cut-off value. Thus one could say "if we set the cut-off to X, we'd select 5% of the cases and get precision of 50% and recall of 10%". Currently the 5% is missing from the picture.

This would support a very common use case (at least in our organization) of setting a threshold to select cases for human intervention based not just on performance (such as precision and recall) but also on resources needed / available (a function of "work rate").

2
2 votes

In the Backlog · Last Updated

Comments

Setup Info
    Tags
      Help me…