Survey banner
The Dataiku Community is moving to a new home! We are temporary in read only mode: LEARN MORE

Function does not reduce error

Level 1
Function does not reduce error


I'm facing some trouble in the following python recipe. 

# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu
from statsmodels.stats.stattools import medcouple

# Read recipe inputs
COLETA_f_datas = dataiku.Dataset("COLETA_f_datas")
COLETA_f_datas_df = COLETA_f_datas.get_dataframe()

# Define relevant functions
def q1(x):
    return x.quantile(0.25)

def q3(x):
    return x.quantile(0.75)
def mc(x):
    y = x[~pd.isnull(x)]
    return medcouple(y)
# Compute recipe outputs from inputs

aggregation_data_compra_df = COLETA_f_datas_df.groupby(['Data', 'ATIVO'])['COMPRA'].agg({'COMPRA_': [q1,q3, mc]}).reset_index()
#aggregation_data_venda_df = COLETA_f_datas_df.groupby(['Data', 'ATIVO'])['VENDA'].agg({'VENDA_': [ q1, q3, medcouple]}).reset_index()
#aggregation_data_indicativa_df = COLETA_f_datas_df.groupby(['Data', 'ATIVO'])['INDICATIVA'].agg({'INDICATIVA_': [ q1, q3, medcouple]}).reset_index()

#Taxas_brutas_stats_df = pd.merge(pd.merge(aggregation_data_compra_df,aggregation_data_venda_df, how ='left'),aggregation_data_indicativa_df, how = 'left')

#Taxas_brutas_stats_df.columns = ['Data', 'ATIVO','TaxaCompra_q1','TaxaCompra_q3','TaxaCompra_MC','TaxaVenda_q1','TaxaVenda_q3','TaxaVenda_MC','TaxaIndicativa_q1','TaxaIndicativa_q3','TaxaIndicativa_MC']

COLETA_F_STATS_df = aggregation_data_compra_df

# Write recipe outputs
COLETA_F_STATS = dataiku.Dataset("COLETA_F_STATS")


By  running it the error from the title appears. However running the function in another plataform shows no errors.  By changing the mc function to the simple medcouple function, the recipe performs well. However, the function medcouple seems to consider null cells and, by doing so, the output offers a wrong answer, forcing me to throw out the original vector and build a new one composed by only valid values. Can anyone assist me?


0 Kudos
3 Replies

Hi @jose_deoliveira ,

The error " Function does not reduce error " may be related to the pandas version used. 

What pandas version are you using in DSS?

What pandas version, are you testing this code externally?

Starting DSS 10.0.4 we  also support Pandas 1.1, Pandas 1.2 and Pandas 1.3. 

0 Kudos
Level 1

Hi Alex,

I'm using version 0.23.4  in DSS and 1.3.5 externally. By creating a new python recipe, i cannot see any options to change it. Maybe it is because I do not have the required permissions to do so? By change the version to 1.3 this effect will be applied in everyother python recipe or exclusivelly in this one?


0 Kudos


You can change the Pandas version in code environment used for this recipe.

You can create a new code env with Pandas 1.0 or higher depending on the DSS version and Python version

you can choose the Core package version: 


Let me know if that helps solve your issue.




0 Kudos


Labels (4)
A banner prompting to get Dataiku