I am trying to build a custom metric like below:
from sklearn.metrics import cohen_kappa_score def quadratic_weighted_kappa(y_valid, y_pred): """ Custom scoring function. Must return a float quantifying the estimator prediction quality. - y_valid is a pandas Series - y_pred is a numpy ndarray with shape: - (nb_records,) for regression problems and classification problems where 'needs probas' (see below) is false (for classification, the values are the numeric class indexes) - (nb_records, nb_classes) for classification problems where 'needs probas' is true - [optional] X_valid is a dataframe with shape (nb_records, nb_input_features) - [optional] sample_weight is a numpy ndarray with shape (nb_records,) NB: this option requires a variable set as "Sample weights" """ metric = cohen_kappa_score(y_valid, y_pred, weights='quadratic') return metric
However I am getting a value error:
Trying to enrich exception: com.dataiku.dip.io.SocketBlockLinkKernelException: Failed to train : <class 'ValueError'> : Custom evaluation function not defined from kernel
Any Help with this would be greatly appreciated.