Best way to check if a Snowflake table exists?

info-rchitect
info-rchitect Registered Posts: 186 ✭✭✭✭✭✭

Hi,

I have a flow where I need to first check if a table exists, if not then run some SQL and create the table. If it does exist, I want to insert into the table only if new records arrive for the source tables that initially created the table.

thx


Operating system used: Windows 10

Answers

  • NN
    NN Neuron, Registered, Neuron 2022, Neuron 2023 Posts: 145 Neuron
    edited July 17

    Hi,
    Hoping i have understood your question correct.

    If you wish to write your own code , you can have a look at the SQL Script recipe
    here you can define you create table statement for snowflake as

    CREATE TABLE IF NOT EXISTS "DB"."SCHEMA"."TABLE"
    (
    "COLUMN1" VARCHAR(10) NULL
    );

    This would create your table only if its doesnt exist. and you can also include an insert statement in the same recipe itself.

    For visual recipe, have a look at the dataiku SYNC recipe with the insert instead of overwrite option.

  • info-rchitect
    info-rchitect Registered Posts: 186 ✭✭✭✭✭✭

    Hi Neuron,

    I really should have been more clear. In my dataiku flow, I would like to have an IF statement that checks if the table exists and then route to different sub-flows depending on the result. I did figure out a way by feeding in a SQL query into my Python recipe as a dataset. But, it would be nice if this sort of conditional could be abstracted out from the Python recipe and into a stand alone dataiku node.

    thx

  • NN
    NN Neuron, Registered, Neuron 2022, Neuron 2023 Posts: 145 Neuron

    Hi,
    Are you using scenarios to control which part of your flow runs ?

    You can have a look at the Run this step conditionally option

    https://doc.dataiku.com/dss/latest/scenarios/step_flow_control.html


  • info-rchitect
    info-rchitect Registered Posts: 186 ✭✭✭✭✭✭

    Yes I am using scenarios, will check out step flow control, thx!

Setup Info
    Tags
      Help me…