Survey banner
The Dataiku Community is moving to a new home! We are temporary in read only mode: LEARN MORE

BigQuery connection

rafael_rosado97
Level 4
BigQuery connection

Hello, Dataiku Team.

If I want to deploy a project on Automation node, but my project has a connection with BigQuery, I need to create the same connection for Automation node, right?

The thing is that I do not have the same options in the automation node as design node.

In design node I have set a section called Path restrictions ----> Bucket and Managed datasets & folders ---> Managed data subpath. But there are not these options in automation node as you can see on images.

 

Do you know how I can set this options?

 

Thank you!!

bq_1.png

bq_2.png


Operating system used: Linux

0 Kudos
1 Reply
Turribeach

I think you are getting confused. Below is my Designer New Google BigQuery connection screen, I am on v12.6.3. There is no concept of "path" in BigQuery connections and you can't create folders in BigQuery either. I think you are confusing the connection with Google Cloud Storage (GCS) which is a bucket connection not a BigQuery connection. And I think you are getting confused because BigQuery needs / uses a Google Cloud Storage for fast writes  to allow new data to be loaded in BigQuery. So a connection to BigQuery usally involves two Dataiku connections, one for Google Cloud Storage and one for BigQuery which uses the Google Cloud Storage for staging data.

Screenshot 2024-06-05 at 23.03.43.png

Screenshot 2024-06-05 at 23.06.29.png

0 Kudos