Programmatically List Tables in Connection

jkonieczny
jkonieczny Registered Posts: 13 ✭✭✭✭
From a Python notebook, is there a way to list all of the tables contained in a Connection?
Tagged:

Answers

  • Jean-Yves
    Jean-Yves Dataiker Posts: 14 Dataiker

    Hi,

    Here is a little script that returns a dictionary of the different connections and a list of datasets contained in those connections. This script uses the DSS Public API (for more in formation on the Public Api see the documentation).



    Note that it won't list tables of a connection that are not datasets in DSS.

    from dataikuapi import DSSClient

    from collections import defaultdict



    api_key = "your-api-key"

    host = "DSS-host"

    client = DSSClient(host, apiKey)

    def get_datasets_per_connection(client):

    datasets_per_connection = defaultdict(list)

    projects = client.list_project_keys()

    def get_connect(data):

    return data.get('params').get('connection')

    for project in projects:

    for dataset in client.get_project(project).list_datasets():

    datasets_per_connection[get_connect(dataset)].append(dataset['name'])



    return datasets_per_connection



    get_datasets_per_connection(client=client)

  • jkonieczny
    jkonieczny Registered Posts: 13 ✭✭✭✭
    Thank you Jean, although I am looking for a solution that does not require the tables to be Datasets in DSS.
Setup Info
    Tags
      Help me…