Dataset status - check error

smiley

Hi, I am new in DSS, when trying to have a check on the dataset column value to ensure each row fall within the numeric range, it getting the error "ARRAY is not numeric", how can this be resolve? Appreciate help here. Thanks.

Regards,
KK

fchataigner2

your metric is indeed an array, since it's set to keep all values of the column. What you want to achieve should rather be done by:

- computing the min and max metrics of the Validation column (in the Column statistics section of the metrics setup)

- add 2 checks: a check that the min value cannot be below -1, and a check that the max value cannot be above 1

fchataigner2

Hi,

the message implies that the value of the metric isn't a numeric as the check expects. Can you check the setup of the "Cell Value" probe, and the recorded values for the metric by selecting "Validation of Cell Value" in the diplayed metrics of the Status tab?

Go14

Hi smiley,

It seems the dataset column which you selected has values which is not numeric data type. For Numeric range check, the values should be a numeric data type.

The Error says "ARRAY is not numeric" i guess your column value is of ARRAY data type. So you can use "Python check" instead of numeric check for checking the column value ranges.

In Python check, you can write your custom python code to do the checks.

I hope this will help you to fix this issue.

smiley

@fchataigner2
Thanks.

The metrics setup as attached. The dataset is having 14rows and need to return error if any of the row value is not within the range -1 to 1. This check status will serve as the decision point of whether the next scenario should run or not. Thanks.

Regards,
KK

smiley

@Go14
I have not write any python code before, tried to refer to Custom probes and checks — Dataiku DSS 8.0 documentation but not able to get it work out, appreciate your help if possible. Thanks.

def process(last_values, dataset):
# last_values is a dict of the last values of the metrics,
# with the values as a dataiku.metrics.MetricDataPoint.
# dataset is a dataiku.Dataset object
#return 'OK', 'optional message' # or 'WARNING' or 'ERROR'
df = dataset.get_dataframe()
val = df['Validation'].tolist()

if val < 1:
return 'OK'
else:
return 'ERROR'

smiley

@fchataigner2
Thanks. This seem work for my requirement.