Creatring a custom scoring metric

Solved!
mkamal
Level 1
Creatring a custom scoring metric

Hello @Alex_Combessie ,

I want to code a custom scoring function for scikit-learn's average-precision-score. It's a for a Keras image binary classification model. I'm not familiar with how the Dataiku API works, but I want to pass the array dimensions to y_valid and y_pred. 

Here's what I have:

from sklearn.metrics import average_precision_score

def score(y_valid, y_pred):
"""
Custom scoring function.
Must return a float quantifying the estimator prediction quality.
- y_valid is a pandas Series
- y_pred is a numpy ndarray with shape:
- (nb_records,) for regression problems and classification problems
where 'needs probas' (see below) is false
(for classification, the values are the numeric class indexes)
- (nb_records, nb_classes) for classification problems where
'needs probas' is true
- [optional] X_valid is a dataframe with shape (nb_records, nb_input_features)
- [optional] sample_weight is a numpy ndarray with shape (nb_records,)
NB: this option requires a variable set as "Sample weights"
"""
average_precision = average_precision_score(y_valid, y_pred)

return average_precision

0 Kudos
1 Solution
Alex_Combessie
Dataiker
Dataiker

Hi,

That seems to be a good piece of code to start with. Are you encountering an error when training? If so, could you post the error message, please?

Hope it helps,

Alex

View solution in original post

2 Replies
Alex_Combessie
Dataiker
Dataiker

Hi,

That seems to be a good piece of code to start with. Are you encountering an error when training? If so, could you post the error message, please?

Hope it helps,

Alex

View solution in original post

mkamal
Level 1
Author

Works great, thanks for verifying

0 Kudos
A banner prompting to get Dataiku DSS
Public