SQL Recipe: Error initializing column metadata

Options
tucel
tucel Partner, Registered Posts: 9 Partner
edited July 16 in Using Dataiku

Hi!

I got an error message while trying to run a sql recipe to create a bigquery table from another ones. I tried the query on bigquery and it's working on there. I am curious about why the same query cannot work on sql recipe in dataiku. Error message:

Validation failed
Query failed: [Simba][BigQueryJDBCDriver](100078) Error initializing column metadata: Could not retrieve table schema information. , caused by: GeneralException: [Simba][BigQueryJDBCDriver](100078) Error initializing column metadata: Could not retrieve table schema information.

I tried to create the table in bigquery and import it into dss in order to dss may see the table schema but nothing changed.
Note: My query might be a little complex but bigquery can handle it. And I choosed the SQL Query option while creating a SQL recipe. Should not it run the query on bigquery engine directly?

Best Answer

  • tucel
    tucel Partner, Registered Posts: 9 Partner
    Answer ✓
    Options

    Hi, thanks for return!

    After several attemps to fix the issue, I realized that SQL recipe could not handle the query with a declare method. I was declaring a variable before the query. After removing this declare method dss could run the recipe. So, issue has fixed

    Regards,

Answers

  • fchataigner2
    fchataigner2 Dataiker Posts: 355 Dataiker
    Options

    Hi,

    the error seems to happen at validation time (which is done before the recipe run to compute the output column types), but the message itself is insufficient to pinpoint a cause. Maybe unsupported column types? You'd need to open a ticket on support.dataiku.com with a diagnostic of the instance, that you can generate in Administration > Maintenance > Diagnostic tool, and send to us via dl.dataiku.com

  • CoreyS
    CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭
    Options

    Thank you for sharing your solution with the rest of the commmunity @tucel
    !

Setup Info
    Tags
      Help me…