I want to create custom scoring metric to measure precision at first 100 predictions (100 highest probability), but as I understand from the comment in score function, I should pass needs_proba = True in order to get 2 dimensional y_pred to get probabilities and use it to sort my predictions. I don't know where I can pass this parameter
def score(y_valid, y_pred):
"""
Custom scoring function.
Must return a float quantifying the estimator prediction quality.
- y_valid is a pandas Series
- y_pred is a numpy ndarray with shape:
- (nb_records,) for regression problems and classification problems
where 'needs probas' (see below) is false
(for classification, the values are the numeric class indexes)
- (nb_records, nb_classes) for classification problems where
'needs probas' is true
- [optional] X_valid is a dataframe with shape (nb_records, nb_input_features)
- [optional] sample_weight is a numpy ndarray with shape (nb_records,)
NB: this option requires a variable set as "Sample weights"
"""
Operating system used: MacOS
