Get Calculated Statistics with Model Evaluation Store API?

Options
cmjurs
cmjurs Registered Posts: 20 ✭✭✭✭

Im aware that you can get a hold of many metrics in the model evaluation store through the use of the API

e.g.

mes = dataiku.ModelEvaluationStore("XXX")
mes_info = mes.get_info()

the_MES_list = mes.get_metric_history("XXX")

My question is, how can I extract the calculated valuesin the MES, like those in the univariate data drift section (image)? Id like to be able to alarm on these values..

Thanks!

CJ


Operating system used: Ubuntu

Tagged:

Best Answer

  • fsergot
    fsergot Dataiker, Registered, Product Ideas Manager Posts: 117 Dataiker
    Answer ✓
    Options

    Hello,

    You need to through the path of Project > Model Evaluation Store > Model Evaluation and in there call compute_data_drift().

    Here is a sample code that does that:

    import dataikufrom dataiku import pandasutils as pduimport pandas as pdimport pprintpp = pprint.PrettyPrinter(indent=4)client = dataiku.api_client()proj = client.get_project("DKU_ENERGY_CONSUMPTION_2")mes = proj.get_model_evaluation_store("b0G6ywDN")print("Found MES '{}'".format(mes.mes_id))me = mes.get_latest_model_evaluation()print("Latest ME is '{}'".format(me.full_id))metrics = me.get_metrics()#pp.pprint(metrics)print("Latest ME metrics are: ")for metric in metrics['metrics'] :print(" {} = {}".format(metric['meta']['name'],metric['lastValues'][0]['value']))print("--- Getting last data drift metrics ---")drift = me.compute_data_drift()pp.pprint(drift.get_raw())univariateDriftResult = drift.get_raw()['univariateDriftResult']['columns']for feature in univariateDriftResult :feat = univariateDriftResult[feature]pp.pprint(feat)print("Feature '{}' => KS test = {} | Chi-square test = {} | PSI = {}".format(feat['name'],feat['ksTestPvalue'] if 'ksTestPvalue' in feat else "N/A",feat['chiSquareTestPvalue'] if 'chiSquareTestPvalue' in feat else "N/A",feat['populationStabilityIndex'] if 'ksTestPvalue' in feat else "N/A",))

Answers

Setup Info
    Tags
      Help me…