Dataset status - check error

smiley
smiley Registered Posts: 6 ✭✭✭✭

Hi, I am new in DSS, when trying to have a check on the dataset column value to ensure each row fall within the numeric range, it getting the error "ARRAY is not numeric", how can this be resolve? Appreciate help here. Thanks.

Regards,
KK

Best Answer

  • fchataigner2
    fchataigner2 Dataiker Posts: 355 Dataiker
    Answer ✓

    your metric is indeed an array, since it's set to keep all values of the column. What you want to achieve should rather be done by:

    - computing the min and max metrics of the Validation column (in the Column statistics section of the metrics setup)

    - add 2 checks: a check that the min value cannot be below -1, and a check that the max value cannot be above 1

Answers

  • fchataigner2
    fchataigner2 Dataiker Posts: 355 Dataiker

    Hi,

    the message implies that the value of the metric isn't a numeric as the check expects. Can you check the setup of the "Cell Value" probe, and the recorded values for the metric by selecting "Validation of Cell Value" in the diplayed metrics of the Status tab?

  • Go14
    Go14 Registered Posts: 5 ✭✭✭

    Hi smiley,


    It seems the dataset column which you selected has values which is not numeric data type. For Numeric range check, the values should be a numeric data type.

    The Error says "ARRAY is not numeric" i guess your column value is of ARRAY data type. So you can use "Python check" instead of numeric check for checking the column value ranges.

    In Python check, you can write your custom python code to do the checks.

    I hope this will help you to fix this issue.

  • smiley
    smiley Registered Posts: 6 ✭✭✭✭

    @fchataigner2
    Thanks.

    The metrics setup as attached. The dataset is having 14rows and need to return error if any of the row value is not within the range -1 to 1. This check status will serve as the decision point of whether the next scenario should run or not. Thanks.

    Regards,
    KK

  • smiley
    smiley Registered Posts: 6 ✭✭✭✭

    @Go14
    I have not write any python code before, tried to refer to Custom probes and checks — Dataiku DSS 8.0 documentation but not able to get it work out, appreciate your help if possible. Thanks.

    def process(last_values, dataset):
    # last_values is a dict of the last values of the metrics,
    # with the values as a dataiku.metrics.MetricDataPoint.
    # dataset is a dataiku.Dataset object
    #return 'OK', 'optional message' # or 'WARNING' or 'ERROR'
    df = dataset.get_dataframe()
    val = df['Validation'].tolist()

    if val < 1:
    return 'OK'
    else:
    return 'ERROR'

  • smiley
    smiley Registered Posts: 6 ✭✭✭✭

    @fchataigner2
    Thanks. This seem work for my requirement.

Setup Info
    Tags
      Help me…