Dataiku DSS API

General API notes

Many API calls may return contain additional attributes compared to the documentation. The stability of these additional attributes is not guaranteed. They may be modified or removed in future releases without notice.
All API calls will return the following two HTTP headers
- DSS-API-Version: Version of the API server handling the request
- DSS-Version : Version of the DSS backend answering the request

Projects¶

List projects
GET`/projects/{tags}`

Lists the projects. Only the projects on which the API key has the READ_CONF privilege will be listed.

🔒 Required privileges : READ_CONF

Parameters

HideShow

tags: string (optional)
Comma separated list of tags. The query will only return projects having one of these tags

Response 200

HideShow

Headers

Content-Type: application/json

Body

[
  {
    "projectKey": "MYPROJECT"
  }
]

Schema

{
  "type": "array",
  "$schema": "http://json-schema.org/draft-04/schema#"
}

Create project
POST`/projects`

Creates a new project

🔒 Required privileges : Admin

Request

HideShow

A project key must match [A-Za-z_]*

Headers

Content-Type: application/json

Body

{
  "projectKey": "The project key of the new project"
}

Response 200

Project¶

Get project metadata
GET`/projects/{projectKey}/metadata`

Retrieves metadata about this project.

🔒 Required privileges : ADMIN (on project)

Parameters

HideShow

projectKey

string (required)

The key of the project to retrieve

NB The project key is usually different than the project displayed name.

The projectKey can be found with the list projects API call or in the URL when accessing DSS GUI.

Response 200

HideShow

Headers

Content-Type: application/json

Body

{
    "label" : "My first project",
    "description" : "This is a sample project summary",
    "shortDesc" : "sample project summary",
    /* It is advised to keep tags as short words */
    "tags" : [
        "my tag 1",
        "my tag 2",
        ...
    ],
    "custom" : {
        "kv" : {
            "as a user" : "I can write",
            "whatever" : [ "I", "want" , "here"]
        }
    }
}

Schema

{"type":"object","properties":{"label":{"type":"string","description":"Display name for this object"},"description":{"type":"string","description":"Long description (Markdown) for this object"},"tags":{"type":"array","description":"Tags of this object"},"custom":{"type":"object","description":"Custom opaque metadata"}},"$schema":"http://json-schema.org/draft-04/schema#"}

Update project metadata
PUT`/projects/{projectKey}/metadata`

Updates metadata about this project. You should only set a metadata object that has been obtained through the corresponding GET call.

🔒 Required privileges : ADMIN (on project)

Parameters

HideShow

projectKey

string (required)

The key of the project to retrieve

NB The project key is usually different than the project displayed name.

The projectKey can be found with the list projects API call or in the URL when accessing DSS GUI.

Request

HideShow

Headers

Content-Type: application/json

Body

{
    "label" : "My first project",
    "description" : "This is a sample project summary",
    "shortDesc" : "sample project summary",
    /* It is advised to keep tags as short words */
    "tags" : [
        "my tag 1",
        "my tag 2",
        ...
    ],
    "custom" : {
        "kv" : {
            "as a user" : "I can write",
            "whatever" : [ "I", "want" , "here"]
        }
    }
}

Schema

{"type":"object","properties":{"label":{"type":"string","description":"Display name for this object"},"description":{"type":"string","description":"Long description (Markdown) for this object"},"tags":{"type":"array","description":"Tags of this object"},"custom":{"type":"object","description":"Custom opaque metadata"}},"$schema":"http://json-schema.org/draft-04/schema#"}

Get project permissions
GET`/projects/{projectKey}/permissions`

Retrieves access permissions for this project.

🔒 Required privileges : ADMIN (on project)

Parameters

HideShow

projectKey

string (required)

The key of the project to retrieve

NB The project key is usually different than the project displayed name.

The projectKey can be found with the list projects API call or in the URL when accessing DSS GUI.

Response 200

HideShow

Headers

Content-Type: application/json

Body

{
    "owner" : "user1",
    "permissions" : [
        {
            "group" : "data_scientists",
            /* The valid permission types are READER, ANALYST_READONLY, ANALYST_READWRITE and ADMIN */
            "type" : "ANALYST_READWRITE"
        }
    ]
}

Schema

{"type":"object","properties":{"owner":{"type":"string","description":"Login of the owner of the proejct"},"permissions":{"type":"array","description":"List of group -> access level mapping"}},"$schema":"http://json-schema.org/draft-04/schema#"}

Update project permissions
PUT`/projects/{projectKey}/permissions`

Updates access permissions for this project. You should only set a permissions object that has been obtained through the corresponding GET call.

🔒 Required privileges : ADMIN (on project)

Parameters

HideShow

projectKey

string (required)

The key of the project to update

NB The project key is usually different than the project displayed name.

The projectKey can be found with the list projects API call or in the URL when accessing DSS GUI.

Request

HideShow

Headers

Content-Type: application/json

Body

{
    "owner" : "user1",
    "permissions" : [
        {
            "group" : "data_scientists",
            /* The valid permission types are READER, ANALYST_READONLY, ANALYST_READWRITE and ADMIN */
            "type" : "ANALYST_READWRITE"
        }
    ]
}

Schema

{"type":"object","properties":{"owner":{"type":"string","description":"Login of the owner of the proejct"},"permissions":{"type":"array","description":"List of group -> access level mapping"}},"$schema":"http://json-schema.org/draft-04/schema#"}

Response 204

Delete project
DELETE`/projects/{projectKey}`

Permanently deletes a project and all its associated datasets, recipes, models, etc.

🔒 Required privileges : ADMIN (on this project)

Parameters

HideShow

projectKey: string (required)
The key of the project to delete

Response 204

Export project
GET`/projects/{projectKey}/export{?exportUploads}{?exportManaged}{?exportAnalysisModels}{?exportSavedModels}`

Exports a whole project configuration and (optionally) its associated data.

Only the content of Managed Filesystem datasets and uploaded datasets can be exported.

The returned zip archive can be used in DSS import feature.

🔒 Required privileges : ADMIN (on this project)

Parameters

HideShow

projectKey: string (required)
The key of the project to delete
exportUploads: boolean (optional) Default: true
Export uploaded datasets data
exportManaged: boolean (optional) Default: true
Export managed Filesystem datasets data
exportAnalysisModels: boolean (optional) Default: true
Export trained models that are in analyses (not deployed in the flow).
exportSavedModels: boolean (optional) Default: true
Export saved models (deployed in the flow).

Response 200

HideShow

Headers

Content-Type: application/zip

Datasets¶

List datasets
GET`/projects/datasets/{?tags}{?foreign}`

Lists the datasets of a project.

🔒 Required privileges : READ_CONF

Parameters

HideShow

foreign: boolean (optional) Default: false
If true, also lists the datasets from other projects that are exposed to the specified project
tags: string (optional)
Comma separated list of tags. The query will only return datasets having one of these tags

Response 200

HideShow

See GET dataset for more information

Headers

Content-Type: application/json

Body

[
  {
    "projectKey": "PKEY1",
    "name": "dataset1",
    "type": "Filesystem",
    "params": {
      "connection": "filesystem_input",
      "path": "/src/dataset1"
    },
    "formatType": "csv",
    "formatParams": {
      "style": "EXCEL",
      "separator": "\t"
    }
  }
]

Schema

{
  "type": "array",
  "$schema": "http://json-schema.org/draft-04/schema#"
}

Create dataset
POST`/projects/datasets`

Create a new dataset.

Important: most parameters and format parameters of datasets are not officially documented and may be modified in future recipes. It is recommended that you use the GET Dataset call to retrieve an existing dataset and modify it to suit your needs and create a new Dataset.

🔒 Required privileges : WRITE_CONF

Request

HideShow

Headers

Content-Type: application/json

Body

{
  "projectKey": "PKEY1",
  "name": "dataset1",
  "type": "Filesystem",
  "params": {
    "connection": "filesystem_input",
    "path": "/src/dataset1"
  },
  "formatType": "csv",
  "formatParams": {
    "style": "EXCEL",
    "separator": "\t"
  }
}

Schema

{
  "type": "object",
  "properties": {
    "projectKey": {
      "type": "string",
      "description": "Project key of this dataset"
    },
    "name": {
      "type": "string",
      "description": "Unique name of this dataset in its project"
    },
    "type": {
      "type": "string",
      "description": "Type of the dataset."
    },
    "params": {
      "type": "object",
      "description": "Parameters of the connection to the data. The available parameters depend on the dataset type"
    },
    "formatType": {
      "type": "string",
      "description": "For file-based datasets, the name of the format"
    },
    "formatParams": {
      "type": "object",
      "description": "For file-based datasets, the parameters of the format. The actual parameters depend on the format type"
    },
    "managed": {
      "type": "boolean",
      "description": "Whether this is a managed dataset. See [DSS documentation](http://doc.dataiku.com/dss/latest/concepts/#managed-and-external-datasets) for an explanation"
    },
    "schema": {
      "type": "object",
      "description": "Schema of this dataset"
    },
    "tags": {
      "type": "array",
      "description": "Tags of this dataset"
    }
  },
  "$schema": "http://json-schema.org/draft-04/schema#"
}

Response 204

Dataset¶

To build a dataset see POST job

Get dataset settings
GET`/projects/datasets`

Retrieves a Dataset object

🔒 Required privileges : READ_CONF

Response 200

HideShow

Headers

Content-Type: application/json

Body

{
  "projectKey": "PKEY1",
  "name": "dataset1",
  "type": "Filesystem",
  "params": {
    "connection": "filesystem_input",
    "path": "/src/dataset1"
  },
  "formatType": "csv",
  "formatParams": {
    "style": "EXCEL",
    "separator": "\t"
  }
}

Schema

{
  "type": "object",
  "properties": {
    "projectKey": {
      "type": "string",
      "description": "Project key of this dataset"
    },
    "name": {
      "type": "string",
      "description": "Unique name of this dataset in its project"
    },
    "type": {
      "type": "string",
      "description": "Type of the dataset."
    },
    "params": {
      "type": "object",
      "description": "Parameters of the connection to the data. The available parameters depend on the dataset type"
    },
    "formatType": {
      "type": "string",
      "description": "For file-based datasets, the name of the format"
    },
    "formatParams": {
      "type": "object",
      "description": "For file-based datasets, the parameters of the format. The actual parameters depend on the format type"
    },
    "managed": {
      "type": "boolean",
      "description": "Whether this is a managed dataset. See [DSS documentation](http://doc.dataiku.com/dss/latest/concepts/#managed-and-external-datasets) for an explanation"
    },
    "schema": {
      "type": "object",
      "description": "Schema of this dataset"
    },
    "tags": {
      "type": "array",
      "description": "Tags of this dataset"
    }
  },
  "$schema": "http://json-schema.org/draft-04/schema#"
}

Update dataset settings
PUT`/projects/datasets`

Updates the settings of a Dataset.

The Dataset object given as parameter in of a PUT call MUST have been previously obtained from a GET dataset call at the same URL.

The object obtained with the GET method may contain undocumented attributes. You should not modify them as it could create an invalid state and thoses attributes may be removed in future releases without notice.

🔒 Required privileges : WRITE_CONF

Request

HideShow

Headers

Content-Type: application/json

Body

{
    "projectKey" : "PKEY1",
    "name" : "dataset1",
    "type" : "Filesystem",
    "params" : {
        "connection" : "filesystem_input",
        "path" : "/src/dataset1"
    },
    "formatType" : "csv",
    "formatParams" : {
        "style" : "EXCEL",
        "separator" : "\t"
    }
    ...
}

Schema

{"type":"object","properties":{"projectKey":{"type":"string","description":"Project key of this dataset"},"name":{"type":"string","description":"Unique name of this dataset in its project"},"type":{"type":"string","description":"Type of the dataset."},"params":{"type":"object","description":"Parameters of the connection to the data. The available parameters depend on the dataset type"},"formatType":{"type":"string","description":"For file-based datasets, the name of the format"},"formatParams":{"type":"object","description":"For file-based datasets, the parameters of the format. The actual parameters depend on the format type"},"managed":{"type":"boolean","description":"Whether this is a managed dataset. See [DSS documentation](http://doc.dataiku.com/dss/latest/concepts/#managed-and-external-datasets) for an explanation"},"schema":{"type":"object","description":"Schema of this dataset"},"tags":{"type":"array","description":"Tags of this dataset"}},"$schema":"http://json-schema.org/draft-04/schema#"}

Response 204

Delete dataset
DELETE`/projects/datasets/{?dropData}`

Deletes a dataset.

WARNING : Deleting a dataset will trigger the deletion of all associated analyses and recipes.

🔒 Required privileges : WRITE_CONF, WRITE_DATA

Parameters

HideShow

dropData: boolean (optional) Default: false
If true, all the data associated with the dataset will also be removed.

Response 204

Dataset metadata¶

Get metadata
GET`/projects/{projectKey}/datasets/{datasetName}/metadata`

Retrieves metadata about this dataset.

🔒 Required privileges : READ_METADATA on Dataset

Parameters

HideShow

projectKey: string (required)
datasetName: string (required)

Response 200

HideShow

Headers

Content-Type: application/json

Body

{
  "label": "dataset_name",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Schema

{
  "type": "object",
  "properties": {
    "label": {
      "type": "string",
      "description": "Display name for this object"
    },
    "description": {
      "type": "string",
      "description": "Long description (Markdown) for this object"
    },
    "tags": {
      "type": "array",
      "description": "Tags of this object"
    },
    "custom": {
      "type": "object",
      "description": "Custom opaque metadata"
    }
  },
  "$schema": "http://json-schema.org/draft-04/schema#"
}

Sets metadata
PUT`/projects/{projectKey}/datasets/{datasetName}/metadata`

Writes metadata about this dataset. You should only set a metadata object that has been obtained through the corresponding GET call.

🔒 Required privileges : WRITE_METADATA on Dataset

Parameters

HideShow

projectKey: string (required)
datasetName: string (required)

Request

HideShow

Headers

Content-Type: application/json

Body

{
  "label": "dataset_name",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Schema

{
  "type": "object",
  "properties": {
    "label": {
      "type": "string",
      "description": "Display name for this object"
    },
    "description": {
      "type": "string",
      "description": "Long description (Markdown) for this object"
    },
    "tags": {
      "type": "array",
      "description": "Tags of this object"
    },
    "custom": {
      "type": "object",
      "description": "Custom opaque metadata"
    }
  },
  "$schema": "http://json-schema.org/draft-04/schema#"
}

Response 200

Dataset schema¶

Get schema
GET`/projects/datasets/schema`

Retrieves the schema of the specified dataset. The dataset’s schema is the list of its columns with their types.

🔒 Required privileges : READ_SCHEMA on Dataset

Response 200

HideShow

The Schema object has one attribute: columns (an array of SchemaColumn) - columns array[Column]

Each SchemaColumn has a name and a type:

name string
type string
maxLength int : for string type only, -1 means no maximum length

Existing types are:

string
boolean
tinyint, smallint, int, bigint
float, double
date
array, map, object
geopoint, geometry

Headers

Content-Type: application/json

Body

{
  "columns": [
    {
      "name": "col0",
      "type": "string",
      "maxLength": 200
    },
    {
      "name": "col1",
      "type": "bigint"
    }
  ]
}

Schema

{
  "type": "object",
  "properties": {
    "columns": {
      "type": "array"
    }
  },
  "$schema": "http://json-schema.org/draft-04/schema#"
}

Set schema
PUT`/projects/datasets/schema`

The Schema object given as parameter in of a PUT call MUST have been previously obtained from a GET schema call at the same URL.

The object with the GET method may contain undocumented attributes. You should not modify them as it could create an invalid state and thoses attributes may be removed in future releases without notice.

🔒 Required privileges : WRITE_SCHEMA

Request

HideShow

The Schema object has one attribute: columns (an array of SchemaColumn) - columns array[Column]

Each SchemaColumn has a name and a type:

name string
type string
maxLength int : for string type only, -1 means no maximum length

Existing types are:

string
boolean
tinyint, smallint, int, bigint
float, double
date
array, map, object
geopoint, geometry

Headers

Content-Type: application/json

Body

{
    columns: [
        {"name": "Column1", type: "string", maxLength: -1},
        {"name": "Column2", type: "bigint"},
        ...
    ]
}

Schema

{"type":"object","properties":{"columns":{"type":"array"}},"$schema":"http://json-schema.org/draft-04/schema#"}

Response 204

Dataset data¶

Get data
GET`/projects/datasets/data{?format}{?columns}{?partitions}{?filter}`

Streams the content of a dataset.

🔒 Required privileges : READ_DATA on Dataset

Parameters

HideShow

format: string (optional) Default: json
Output format name
columns: string (optional) Example: mycol1,mycol2
List of requested columns, as a comma-separated list
partitions: string (optional) Example: 2015-07-07
Partition list specification
filter: string (optional) Example: mycol1 > 0 && mycol3 > 0
Formula to select only a subset of rows based on a boolean expression

Response 200

HideShow

The Content-Type and the content of the request may change according to the requested format.

The default (json) output will produce an array of arrays representing the data:

Body

[
    [ "a", "1"],
    [ "b", "2"],
    ...
]

Get data - alternative version
POST`/projects/datasets/data`

Streams the content of a dataset.

This is a variant of the previous method using POST to post large and complex requests

🔒 Required privileges : READ_DATA

Request

HideShow

Headers

Content-Type: application/json

Body

{
  "format": "json",
  "sampling": {
    "samplingMethod": "HEAD_SEQUENTIAL",
    "maxRecords": 30000
  },
  "columns": [
    "col1",
    "col2"
  ],
  "partitions": "2015-03-01|2015-05-04",
  "filterExpression": "mycol1 > 0 && mycol3 > 0"
}

Schema

{
  "type": "object",
  "properties": {
    "columns": {
      "type": "array",
      "description": "If null, all columns are returned"
    },
    "partitions": {
      "type": "string",
      "description": "Partition spec"
    }
  },
  "$schema": "http://json-schema.org/draft-04/schema#"
}

Response 200

HideShow

The Content-Type and the content of the request may change according to the requested format.

The default (json) output will produce an array of arrays representing the data:

Body

[
    [ "a", "1"],
    [ "b", "2"],
    ...
]

Clear data
DELETE`/{?partitions}`

Clears the data contained in the dataset; the dataset itself is not deleted.

If a list of partition identifiers is specified, only the corresponding partitions are cleared.

🔒 Required privileges : WRITE_DATA

Parameters

HideShow

partitions: string) - List of partitions to clear (as a partitions spec (optional)

Response 204

Partitions¶

List partitions
GET`/projects/datasets/partitions`

Lists the partitions of the specified dataset. If the dataset is not partitioned, returns ["NP"]

🔒 Required privileges : READ_DATA

Response 200

HideShow

Body

[
        "2015-01-01",
        "2015-01-02",
        ...
    ]

Jobs¶

List latest jobs
GET`/projects/jobs{?limit}`

Retrieves the list of the last jobs, as an array of JobSummary objects as defined in in the GET job call.

🔒 Required privileges : READ_CONF

Parameters

HideShow

limit: int (optional) Default: 0
Maximum number of returned jobs, 0 for no limit.

Response 200

HideShow

Headers

Content-Type: application/json

Body

array[JobSummary]

Schema

{"type":"array","$schema":"http://json-schema.org/draft-04/schema#"}

Run job
POST`/projects/jobs`

Start building a list of outputs.

A successful call means that the job was successfully initilized, not that it is completed. To follow the build progress use GET job.

Returns the complete job definition, including identifier of the job

🔒 Required privileges : RUN_JOBS

Request

HideShow

Headers

Content-Type: application/json

Body

{
    "outputs": [
        {
            "id" : "MyDatasetName"
        },
    ]
}

Schema

{"type":"object","properties":{"outputs":{"type":"array","description":"Outputs to build for the job"},"type":{"type":"string","description":"Type of job to build. One of RECURSIVE_BUILD, NON_RECURSIVE_FORCED_BUILD, RECURSIVE_FORCED_BUILD, RECURSIVE_MISSING_ONLY_BUILD"}},"required":["outputs"],"$schema":"http://json-schema.org/draft-04/schema#"}

Response 200

HideShow

Body

{
    "projectKey" : "MYPROJECT",
    "id" : "azeop12309xolsk",
}

Job¶

Get job status
GET`/projects/jobs`

Retrieves the job status as a JobSummary object summarising the state of a job and its activities.

A DSS job is a sequence of operations performed on datasets.

A job is divided into activities, each activity corresponds to a recipe run using a given set of partitions.

Example : running two recipes on 2 partitions will result in a job with 4 activities.

🔒 Required privileges : MONITOR_JOBS

Response 200

HideShow

Headers

Content-Type: application/json

Body

{
    "baseStatus" : {
        /* Possible status are NOT_STARTED, RUNNING, FAILED, ABORTED and DONE */
        "status" : "DONE",
        "jobStartTime":1442418929502,
        "jobEndTime":1442418932140
    }
}
Requests the specified job to abort.

**NB** It may take some time for the job to actually stop, so
when the servers answers this request, the job cannot be considered stopped.

Response 204

Job logs¶

Get job logs
GET`/projects/jobs/log{?activity}`

Retrieves the logs content for a job or one of its activities (if specified).

🔒 Required privileges : READ_CONF

Parameters

HideShow

activity: string (optional)

Response 200

HideShow

Headers

Content-Type: text/plain

SQL queries¶

Start a query
POST`/sql/queries`

Start the execution of a query.

This call starts the execution of the query and returns the result’s schema, along with an identifier for the query.

🔒 Required privileges : RUN_CODE

Request

HideShow

Headers

Content-Type: application/json

Body

{
  "connection": "myvertica",
  "query": "SELECT COUNT(*) from bigtable"
}

Schema

{
  "type": "object",
  "properties": {
    "connection": {
      "type": "string",
      "description": "If set, the query is run on the database configured on the connection. If the connection is a HDFS one, then the hive database configured on the connection is used"
    },
    "datasetFullName": {
      "type": "string",
      "description": "If set, the query is run on the database used by the dataset. Dataset must be given as a full name (PROJECTKEY.datasetName)"
    },
    "database": {
      "type": "string",
      "description": "(only for Hive and Impala queries) If set, the query is run on this Hive database"
    },
    "query": {
      "type": "string",
      "description": "A SQL query"
    },
    "preQueries": {
      "type": "array",
      "description": "A list of queries to run before the actual query",
      "default": "empty"
    },
    "postQueries": {
      "type": "array",
      "description": "A list of queries to run after the actual query",
      "default": "empty"
    },
    "type": {
      "type": "string",
      "description": "The type of database  the query is run on. Can be `sql`, `hive` or `impala`."
    }
  },
  "$schema": "http://json-schema.org/draft-04/schema#"
}

Response 200

HideShow

Headers

Content-Type: application/json

Body

{
    "queryId" : "e98f95f2-678b-4277-ac01-c5ff52b8cd9a",
    "hasResults" : true,
    "schema" : [
        {
            "name" : "col_1",
            "type": "string"
        },
        {
            "name" : "col_2",
            "type": "int"
        },
        ...
    ]
}

Schema

{"type":"object","$schema":"http://json-schema.org/draft-04/schema#"}

Stream the data
GET`/sql/queries/{queryId}/stream{?format}{?formatParams}`

Streams the results of a query.

🔒 Required privileges : RUN_CODE

Parameters

HideShow

queryId: string (required)
Identifier returned by the start-streaming call
format: string (optional) Default: json
Output format name
formatParams: object (optional)
Output format parameters (depends on the format)

Response 200

HideShow

The Content-Type and the content of the request may change according to the requested format.

The default (json) output will produce an array of arrays representing the data:

Body

[
    [ "a", "1"],
    [ "b", "2"],
    ...
]

Verify a query
GET`/sql/queries/{queryId}/finish-streaming`

Start the execution of a query

🔒 Required privileges : RUN_CODE

Parameters

HideShow

queryId: string (required)
Identifier returned by the start-streaming call

Response 200

HideShow

Verifies that the query identified by queryId finished successfully on the server side. If not, returns the exception and a 500 status code.

Headers

Content-Type: application/text

Body

exception

Connections¶

List connections
GET`/admin/connections`

List all the connections available on the DSS instance.

This call retrieves a dictionary of Connection objects as defined in GET connection.

🔒 Required privileges : Admin

Response 200

HideShow

Headers

Content-Type: application/json

Body

{
    "my-connection": {
        "name" : "my-connection",
        "type" : "Vertica",
        "allowWrite" : true,
        "allowManagedDatasets" : true,
        "usableBy" : "ALL",
        "allowedGroups" : [],
        "params" : {
            "db" : "dbname",
            "properties" : {},
            "user" : "dbadmin",
            "host" : "127.0.0.1",
            "password" : "thedbpassword"
        },
    }
}

Create connection
POST`/admin/connections`

Creates a connection on DSS.

The parameters of a connection are specific to each type of connection. It is recommended that you have a look at the parameters of a similar connection to create your own.

🔒 Required privileges : Admin

Request

HideShow

Headers

Content-Type: application/json

Body

{
  "name": "new-connection",
  "type": "PostgreSQL",
  "params": {
    "db": "dbname",
    "user": "myuser",
    "host": "127.0.0.1",
    "password": "thedbpassword"
  }
}

Response 200

Connection¶

🔒 Required privileges : Admin

Get connection
GET`/admin/connection`

Gets a connection

WARNING : The returned object may contain other attributes but you should not rely on them since they could be modified or removed in future releases without notice.

Response 200

HideShow

Headers

Content-Type: application/json

Body

{
  "type": "MySQL",
  "name": "LocalMySQL",
  "allowWrite": true,
  "allowManagedDatasets": true,
  "usableBy": "ALL",
  "allowedGroups": [],
  "params": {
    "db": "dbname",
    "user": "myuser",
    "host": "127.0.0.1",
    "password": "thedbpassword"
  }
}

Schema

{
  "type": "object",
  "properties": {
    "name": {
      "type": "string"
    },
    "type": {
      "type": "string"
    },
    "allowWrite": {
      "type": "boolean"
    },
    "allowManagedDatasets": {
      "type": "boolean",
      "description": "Set to true to allow the connection to provide [Managed datasets](http://doc.dataiku.com/dss/latest/concepts/#managed-and-external-datasets)"
    },
    "usableBy": {
      "type": "string",
      "description": "\"ALL\" or \"ALLOWED\", who may use this connection"
    },
    "allowedGroups": {
      "type": "array",
      "description": "Ignored if usableBy is ALL"
    }
  },
  "$schema": "http://json-schema.org/draft-04/schema#"
}

Update connection
PUT`/admin/connection`

The Connection object given as parameter in of a PUT call MUST have been previously obtained from a GET connection call at the same URL.

**WARNING : ** the type and names attributes may not be modified. The object obtained with the GET method may contain undocumented attributes. You should not modify them as it could create an invalid state and thoses attributes may be removed in future releases without notice.

🔒 Required privileges : Admin

Request

HideShow

Headers

Content-Type: application/json

Body

{
  "type": "MySQL",
  "name": "LocalMySQL",
  "allowWrite": true,
  "allowManagedDatasets": true,
  "usableBy": "ALL",
  "allowedGroups": [],
  "params": {
    "db": "dbname",
    "user": "myuser",
    "host": "127.0.0.1",
    "password": "thedbpassword"
  }
}

Response 204

Delete connection
DELETE`/admin/connection`

Removes the connection from DSS.

WARNING : No check is performed to ensure that the connection is not in use for a dataset.

🔒 Required privileges : Admin

Response 204

Security¶

Users¶

List users
GET`/admin/users/{?connected}`

Retrieves the list of DSS users as a list of User objects as defined in GET user.

Parameters

HideShow

connected: boolean (optional) Default: false
If set to true, only lists the currently connected users.

Response 200

HideShow

Headers

Content-Type: application/json

Body

[
    {
        "login" : "admin",
        "sourceType" : "LOCAL", // LOCAL or LDAP
        "displayName" : "The administrator",
        "groups" : ["administrators", "data_scientists"]
    }
]

Schema

{"type":"array","$schema":"http://json-schema.org/draft-04/schema#"}

Create user
POST`/admin/users`

Adds a user account on DSS.

Request

HideShow

Headers

Content-Type: application/json

Body

{
  "login": "newlogin",
  "displayName": "The new user",
  "groups": [
    "data_scientists"
  ],
  "password": "unencrypted password"
}

Response 200

User¶

Get user
GET`/admin/users`

Retrieves a User object describing a DSS user

WARNING : The returned object may contain other attributes but you should not rely on them since they could be modified or removed in future releases without notice.

🔒 Required privileges : Admin

Response 200

HideShow

Headers

Content-Type: application/json

Body

{
    "login" : "mattsco",
    "displayName" : "Matthieu",
    "groups" : [
        "administrators",
        "data_scientists"
    ],
    "codeAllowed" : true,
}

Schema

{"type":"object","properties":{"login":{"type":"string"},"sourceType":{"type":"string"},"displayName":{"type":"string"},"groups":{"type":"array"},"codeAllowed":{"type":"string","description":"True if the user is allowed to write native code"}},"$schema":"http://json-schema.org/draft-04/schema#"}

Update user
PUT`/admin/users`

The User object given as body of the PUT call MUST have been previously obtained from a GET user call at the same URL.

🔒 Required privileges : Admin

Request

HideShow

Headers

Content-Type: application/json

Body

{
  "login": "mattsco",
  "displayName": "Matthieu Modified",
  "groups": [
    "administrators",
    "data_scientists"
  ],
  "codeAllowed": true
}

Response 204

Delete user
DELETE`/admin/users`

Deletes a DSS user

🔒 Required privileges : Admin

Response 204

Groups¶

List groups
GET`/admin/groups`

Retrieves the list of DSS groups as a list of Group objects as defined in GET group.

🔒 Required privileges : Admin

Response 200

HideShow

Headers

Content-Type: application/json

Body

[
  {
    "name": "administrators",
    "description": "DSS administrators",
    "admin": true,
    "sourceType": "LOCAL"
  }
]

Create group
POST`/admin/groups`

Add a user group to DSS

🔒 Required privileges : Admin

Request

HideShow

Headers

Content-Type: application/json

Body

{
  "name": "New Group for business",
  "admin": false,
  "description": "This group is for business users"
}

Response 200

Group¶

Get group
GET`/admin/groups`

Retrieves a Group object describing a DSS user group, used for access control on connections and projects.

WARNING : The returned object may contain other attributes but you should not rely on them since they could be modified or removed in future releases without notice.

🔒 Required privileges : Admin

Response 200

HideShow

Headers

Content-Type: application/json

Body

{
  "name": "ANALYSTS",
  "admin": false,
  "description": "This group is suited for business analysts"
}

Schema

{
  "type": "object",
  "properties": {
    "name": {
      "type": "string"
    },
    "description": {
      "type": "string"
    },
    "admin": {
      "type": "boolean",
      "description": "Whether this group gives administrative rights on DSS"
    },
    "sourceType": {
      "type": "string"
    }
  },
  "$schema": "http://json-schema.org/draft-04/schema#"
}

Update group
PUT`/admin/groups`

The Group object given as parameter in of a PUT call MUST have been previously obtained from a GET group call at the same URL.

🔒 Required privileges : Admin

Request

HideShow

Headers

Content-Type: application/json

Body

{
  "name": "ANALYSTS",
  "admin": false,
  "description": "This group is suited for business analysts"
}

Response 204

Delete group
DELETE`/admin/groups`

Deletes a DSS users group

🔒 Required privileges : Admin

Response 204

DSS administration¶

Platform logs¶

List logs
GET`/admin/logs`

DSS uses a number of log files, for example for the web server, the notebooks or the core backend plateform.

This call list all the available logs but NOT the logs generated during the jobs execution.

For these, see the Jobs section.

🔒 Required privileges : Admin

Response 200

HideShow

Headers

Content-Type: application/json
DSS-API-Version: Version of the API server handling the request
DSS-Version: Version of the DSS backend answering the request

Body

{
    "logs" : [
        {
           "name" : "access.log",
           "totalSize" : 571166942,
           "lastModified" : 1435840900000
        },
        ...
    ]
}

Log¶

Get log content
GET`/admin/logs`

Retrieves the log file with the specified name.

🔒 Required privileges : Admin

Response 200

HideShow

Headers

Content-Type: text/plain
DSS-API-Version: Version of the API server handling the request
DSS-Version: Version of the DSS backend answering the request

Global variables¶

List variables
GET`/admin/variables`

Retrieves the DSS instance global variable as a dictionary.

🔒 Required privileges : Admin

Response 200

HideShow

Headers

Content-Type: application/json

Body

{
    "variable_1": "value",
    ...
}

Save variables
PUT`/admin/variables`

Save the global variables for the DSS instance.

WARNING : this will overwrite all previous variables, so to update or add only some variables, you should first list the current variables with a GET call.

🔒 Required privileges : Admin

Request

HideShow

Headers

Content-Type: application/json

Body

{
    "variable_1": "value",
    ...
}

Response 204

Projects¶

Projects¶

List projectsGET/projects/{tags}

Headers

Body

Schema

Create projectPOST/projects

Headers

Body

Project¶

Get project metadataGET/projects/{projectKey}/metadata

Headers

Body

Schema

Update project metadataPUT/projects/{projectKey}/metadata

Headers

Body

Schema

Get project permissionsGET/projects/{projectKey}/permissions

Headers

Body

Schema

Update project permissionsPUT/projects/{projectKey}/permissions

Headers

Body

Schema

Delete projectDELETE/projects/{projectKey}

Export projectGET/projects/{projectKey}/export{?exportUploads}{?exportManaged}{?exportAnalysisModels}{?exportSavedModels}

Headers

Datasets¶

Datasets¶

List datasetsGET/projects/datasets/{?tags}{?foreign}

Headers

Body

Schema

Create datasetPOST/projects/datasets

Headers

Body

Schema

Dataset¶

Get dataset settingsGET/projects/datasets

Headers

Body

Schema

Update dataset settingsPUT/projects/datasets

Headers

Body

Schema

Delete datasetDELETE/projects/datasets/{?dropData}

Dataset metadata¶

Get metadataGET/projects/{projectKey}/datasets/{datasetName}/metadata

Headers

Body

Schema

Sets metadataPUT/projects/{projectKey}/datasets/{datasetName}/metadata

Headers

Body

Schema

Dataset schema¶

Get schemaGET/projects/datasets/schema

Headers

Body

Schema

Set schemaPUT/projects/datasets/schema

Headers

Body

Schema

Dataset data¶

Get dataGET/projects/datasets/data{?format}{?columns}{?partitions}{?filter}

Body

Get data - alternative versionPOST/projects/datasets/data

Headers

Body

Schema

Body

Clear dataDELETE/{?partitions}

Partitions¶

List partitionsGET/projects/datasets/partitions

Body

Jobs¶

List projects
GET`/projects/{tags}`

Create project
POST`/projects`

Get project metadata
GET`/projects/{projectKey}/metadata`

Update project metadata
PUT`/projects/{projectKey}/metadata`

Get project permissions
GET`/projects/{projectKey}/permissions`

Update project permissions
PUT`/projects/{projectKey}/permissions`

Delete project
DELETE`/projects/{projectKey}`

Export project
GET`/projects/{projectKey}/export{?exportUploads}{?exportManaged}{?exportAnalysisModels}{?exportSavedModels}`

List datasets
GET`/projects/datasets/{?tags}{?foreign}`

Create dataset
POST`/projects/datasets`

Get dataset settings
GET`/projects/datasets`

Update dataset settings
PUT`/projects/datasets`

Delete dataset
DELETE`/projects/datasets/{?dropData}`

Get metadata
GET`/projects/{projectKey}/datasets/{datasetName}/metadata`

Sets metadata
PUT`/projects/{projectKey}/datasets/{datasetName}/metadata`

Get schema
GET`/projects/datasets/schema`

Set schema
PUT`/projects/datasets/schema`

Get data
GET`/projects/datasets/data{?format}{?columns}{?partitions}{?filter}`

Get data - alternative version
POST`/projects/datasets/data`

Clear data
DELETE`/{?partitions}`

List partitions
GET`/projects/datasets/partitions`

List latest jobs
GET`/projects/jobs{?limit}`

Run job
POST`/projects/jobs`

Get job status
GET`/projects/jobs`

Get job logs
GET`/projects/jobs/log{?activity}`

Start a query
POST`/sql/queries`

Stream the data
GET`/sql/queries/{queryId}/stream{?format}{?formatParams}`

Verify a query
GET`/sql/queries/{queryId}/finish-streaming`

List connections
GET`/admin/connections`

Create connection
POST`/admin/connections`

Get connection
GET`/admin/connection`

Update connection
PUT`/admin/connection`

Delete connection
DELETE`/admin/connection`

List users
GET`/admin/users/{?connected}`

Create user
POST`/admin/users`

Get user
GET`/admin/users`

Update user
PUT`/admin/users`

Delete user
DELETE`/admin/users`

List groups
GET`/admin/groups`