New to Dataiku DSS? Try out our NEW Quick Start Programs today and get onboarded on the product in just one hour! Let's go

Dataset status - check error

Solved!
smiley
Level 1
Dataset status - check error

Hi, I am new in DSS, when trying to have a check on the dataset column value to ensure each row fall within the numeric range, it getting the error "ARRAY is not numeric", how can this be resolve? Appreciate help here. Thanks.

Regards,
KK

0 Kudos
1 Solution
fchataigner2
Dataiker
Dataiker

your metric is indeed an array, since it's set to keep all values of the column. What you want to achieve should rather be done by:

- computing the min and max metrics of the Validation column (in the Column statistics section of the metrics setup)

- add 2 checks: a check that the min value cannot be below -1, and a check that the max value cannot be above 1

View solution in original post

6 Replies
fchataigner2
Dataiker
Dataiker

Hi,

the message implies that the value of the metric isn't a numeric as the check expects. Can you check the setup of the "Cell Value" probe, and the recorded values for the metric by selecting "Validation of Cell Value" in the diplayed metrics of the Status tab?

0 Kudos
smiley
Level 1
Author

@fchataigner2 Thanks.

The metrics setup as attached. The dataset is having 14rows and need to return error if any of the row value is not within the range -1 to 1. This check status will serve as the decision point of whether the next scenario should run or not. Thanks.

Regards,
KK

0 Kudos
fchataigner2
Dataiker
Dataiker

your metric is indeed an array, since it's set to keep all values of the column. What you want to achieve should rather be done by:

- computing the min and max metrics of the Validation column (in the Column statistics section of the metrics setup)

- add 2 checks: a check that the min value cannot be below -1, and a check that the max value cannot be above 1

View solution in original post

smiley
Level 1
Author

@fchataigner2 Thanks. This seem work for my requirement. 🙂

0 Kudos
Go14
Level 1

Hi smiley,

 
It seems the dataset column which you selected has values which is not numeric data type. For Numeric range check, the values should be a numeric data type. 

The Error says "ARRAY is not numeric" i guess your column value is of ARRAY data type. So you can use "Python check" instead of numeric check for checking the column value ranges.

In Python check, you can write your custom python code to do the checks. 

I hope this will help you to fix this issue.

0 Kudos
smiley
Level 1
Author

@Go14 I have not write any python code before, tried to refer to Custom probes and checks — Dataiku DSS 8.0 documentation but not able to get it work out, appreciate your help if possible. Thanks.

def process(last_values, dataset):
# last_values is a dict of the last values of the metrics,
# with the values as a dataiku.metrics.MetricDataPoint.
# dataset is a dataiku.Dataset object
#return 'OK', 'optional message' # or 'WARNING' or 'ERROR'
df = dataset.get_dataframe()
val = df['Validation'].tolist()

if val < 1:
return 'OK'
else:
return 'ERROR'

0 Kudos
A banner prompting to get Dataiku DSS