BigQuery connection
Hello, Dataiku Team.
If I want to deploy a project on Automation node, but my project has a connection with BigQuery, I need to create the same connection for Automation node, right?
The thing is that I do not have the same options in the automation node as design node.
In design node I have set a section called Path restrictions ----> Bucket and Managed datasets & folders ---> Managed data subpath. But there are not these options in automation node as you can see on images.
Do you know how I can set this options?
Thank you!!
Operating system used: Linux
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,090 Neuron
I think you are getting confused. Below is my Designer New Google BigQuery connection screen, I am on v12.6.3. There is no concept of "path" in BigQuery connections and you can't create folders in BigQuery either. I think you are confusing the connection with Google Cloud Storage (GCS) which is a bucket connection not a BigQuery connection. And I think you are getting confused because BigQuery needs / uses a Google Cloud Storage for fast writes to allow new data to be loaded in BigQuery. So a connection to BigQuery usally involves two Dataiku connections, one for Google Cloud Storage and one for BigQuery which uses the Google Cloud Storage for staging data.