Meet "Connection xxx not found" when exporting a project
Thanks for your time at the beginning.
I am currently exporting a project, only check the 4 default options.
However, it failed with a warning "An invalid argument has been encountered : Connection 'SF_VAW_PROD_ATP_MED' does not exist"Then I tried to use Python API to find this connection, but failed again:
import dataiku
def get_project(client, project_key):
return client.get_project(project_key)
def find_connection(project, target_connection) -> str:
for dataset in project.list_datasets(as_type='objects'):
ds_settings = dataset.get_settings()
current_connection = ds_settings.get_raw().get('params').get('connection')
if current_connection == target_connection:
return dataset.name
return "not found"
if __name__ == "__main__":
# Dataiku
url = "https://xxxxxxxxxxxxxxxxxxxxx"
api_key = "dkuaps-xxxxxxxxxxxxxxxxxxx"
target_connection = "MY_CONNECTION"
# set up client
dataiku.set_remote_dss(url, api_key, no_check_certificate=True)
client = dataiku.api_client()
# project key
project_key = "MY_PROJECT"
print("project_key: " + project_key)
# project
project = client.get_project(project_key)
# function call
dataset_name = find_connection(project, target_connection)
print("Dataset: " + dataset_name) ===========================================================================================
Result:
project_key: MY_PROJECT Dataset: not found
Do you have any ideas? Appreciate your help a lot.
Operating system used: Windows 11 Enterprise
Best Answer
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,362 DataikerHi,
It appears that the connection 'SF_VAW_PROD_ATP_MED' has been deleted, and some datasets are still referencing it, so the export fails.
To overcome this, you temporarily re-create the connection ( it doesn't have to be working, don't add any credentials, just have the same name)
Next, determine which datasets are still referencing this connection. Per-connection data in (admin/monitoring/connection-data/) changes the connection for the datasets from the flow; re-export the project.
You can finally delete the connection after all objects have been moved to a new connection.
Thanks
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,362 DataikerAdditionally there may be other object beyond datasets that are using the connection like SQL Noteboooks which you may need to check
