Notify if dataset is NON empty

Options
pnaik1
pnaik1 Registered Posts: 23 ✭✭✭✭

Hi all,

I have final dataset with 5 columns (col1, col2, col3, col4, col5). After running whole workflow (using scenario) I want to make sure that 3 columns (col1, col3, col5) of final dataset should not contain null values. In case any of these 3 column value(s) is null then I want to get notified through mail.

Can someone help me with how to proceed.

Thanks in advance!

Best Answer

  • Grixis6
    Grixis6 Registered Posts: 15 ✭✭✭✭✭
    Answer ✓
    Options

    Hello,

    Of course, you can do this in several ways.

    If you simply want to be informed by email at each build of your dataframe if the columns (col1, col3, col5)> not contain null values.

    You can create in the status and metrics section a check on each column if they are not null.

    > If I understood your need correctly: you could include in your scenario 2 more steps after your build: do the checks, send an email (if or not) with the results of your checks attached: empty / not empty for your columns.

    For example; if your check failed that means : your columns are empty.

Answers

  • Ignacio_Toledo
    Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 411 Neuron
    Options

    Hi @pnaik1
    ,

    If you need more material to follow @Grixis6
    suggestion, you can find these sections of the Knowledge Base to be useful:

    1. Automation: Metrics and Checks
    2. Automation: Scenarios

    They have examples on how to create Checks and use them in scenarios. If you'd like a more step-by-step example, please let me know, and I can share some material I've with you.

    Cheers

Setup Info
    Tags
      Help me…