Meaning of pre-join computed columns

Options
Pizarro75
Pizarro75 Partner, Registered Posts: 7 Partner

Hello, I am in charge of migrating some data quality control pipeline that are currently running on Dataiku to Google Cloud using SQL. I am currently on a JOIN RECIPE job and there is one step that confuse me, it is the "pre-join computed columns" as you can see over here:

Screenshot 2021-11-08 at 13.16.57.png

This is what pre-join computed columns contains:

Screenshot 2021-11-08 at 13.19.06.png

And if I open FCST_filter or FCSTDRAFT_filter, I am getting the same thing:

Screenshot 2021-11-08 at 13.19.19.png

What I was expecting is to have a new column for each table (FCST_filter or FCSTDRAFT_filter,) called exists with the expect same value for every row, 1. But in fact, I do have some 0 and some 1 on my output table for the exist column. Do you have some idea where does this number come from?

Screenshot 2021-11-08 at 13.24.48.png

Answers

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 Dataiker
    Options

    Hi,

    Based on the information it looks like the pre-computed would be doing a "SELECT 1 FROM Table"

    Which should return the constant 1 for every row of the table if the record matches your where clause and/or join. If it doesn't match it return nothing. Perhaps those are the 0's you are seeing not sure if they are directly or perhaps added later in your flow.

    See this article for more information.

Setup Info
    Tags
      Help me…