Can I delete or replace a dataset without deleting all the stream ?

Options
BadrFt
BadrFt Registered Posts: 1 ✭✭✭✭
I have an old database (Oracle) that I changed to Vertica, and I have many flows that use the Oracle db tables and I want to replace only the the dataset without losing all the stream.
Tagged:

Answers

  • Clément_Stenac
    Clément_Stenac Dataiker, Dataiku DSS Core Designer, Registered Posts: 753 Dataiker
    edited July 17
    Options

    Hi,

    Most recipes, but not yet all, support replacing an input or an output.

    The best solution, though not officially supported, is to use our API to change the settings of the existing datasets.

    IMPORTANT NOTE: Generally speaking, just changing the type of a dataset will not magically work. However, in the specific case of SQL datasets, they are fairly easily substitutable: you just need to change the "type" and "connection" params.

    The following code (which can run in a notebook) would do this (you have to go to Settings to add a new API key)


    from dataikuapi.dssclient import DSSClient

    host= "http://localhost:10000"
    apiKey="vrKz9rN6L8CA313Cq0719cs0ht7LYohR"
    project="PROJECT"

    client = DSSClient(host, apiKey)
    p = client.get_project(project)

    for dataset in p.list_datasets():
    print "Checking dataset %s" % dataset["name"]
    d = p.get_dataset(dataset["name"])

    definition = d.get_definition()

    if definition["type"] == "Oracle":
    definition["type"] = "Vertica"
    definition["params"]["connection"] = "my_vertica_connection"
    print "Updating dataset %s" % dataset["name"]
    d.set_definition(definition)

Setup Info
    Tags
      Help me…