Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Anyone have a script that lists all the connections used within a project?
Hi @sylvyr3
You will need to go through all the datasets and retrieve their connections:
import dataiku
client = dataiku.api_client()
project = client.get_project(dataiku.default_project_key())
all_datasets = project.list_datasets()
for dataset in all_datasets:
try:
print('Dataset', dataset["name"], 'is using', dataset["params"]["connection"], "connection")
except KeyError:
print('Dataset', dataset["name"], 'is using unlisted connection. Possibly upload dataset')
Hi @sylvyr3
You should be able to do this in the same loop:
import dataiku
client = dataiku.api_client()
project = client.get_project(dataiku.default_project_key())
all_datasets = project.list_datasets()
for d in all_datasets:
dataset = project.get_dataset(d["name"])
definition = dataset.get_definition()
# Getting only datasets that have a connection
try:
connection = definition["params"]["connection"]
except KeyError:
connection = "unlisted connection"
# Check the connection to see if it's the one we need to update
# If it is, modify the dataset connection to map to the new one
if connection == "OLD_CONNECTION":
definition["params"]["connection"] = "NEW_CONNECTION"
dataset.set_definition(definition)
Hi @sylvyr3
You will need to go through all the datasets and retrieve their connections:
import dataiku
client = dataiku.api_client()
project = client.get_project(dataiku.default_project_key())
all_datasets = project.list_datasets()
for dataset in all_datasets:
try:
print('Dataset', dataset["name"], 'is using', dataset["params"]["connection"], "connection")
except KeyError:
print('Dataset', dataset["name"], 'is using unlisted connection. Possibly upload dataset')
Thank you, this was helpful.
I should have asked in the original post, but would it be possible to programmatically change references from connection A to connection B?
I need to create a new connection profile and switch some of the connections to this new profile. I'd rather write a script to do this so I don't miss any references.
Hi @sylvyr3
You should be able to do this in the same loop:
import dataiku
client = dataiku.api_client()
project = client.get_project(dataiku.default_project_key())
all_datasets = project.list_datasets()
for d in all_datasets:
dataset = project.get_dataset(d["name"])
definition = dataset.get_definition()
# Getting only datasets that have a connection
try:
connection = definition["params"]["connection"]
except KeyError:
connection = "unlisted connection"
# Check the connection to see if it's the one we need to update
# If it is, modify the dataset connection to map to the new one
if connection == "OLD_CONNECTION":
definition["params"]["connection"] = "NEW_CONNECTION"
dataset.set_definition(definition)