API Deployer - Lookup endpoint - multiple rows

MoseleyDanielCr
MoseleyDanielCr Registered Posts: 2

I am working on deploying an API on Dataiku with a dataset lookup endpoint. I can retrieve individual rows this way using the API, but I'd like to get multiple rows or even the whole dataset. Is this possible?

Best Answer

  • VitaliyD
    VitaliyD Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer Posts: 102 Dataiker
    edited July 17 Answer ✓

    Hi,

    API node is a separate instance, so to access the dataset from the Design node you would need to use Dataiku API the remote way. Please refer to an example below:

    import dataiku
    from dataiku import pandasutils as pdu
    import pandas as pd
    
    host="dss_host_url:port" # here replace with the DSS host_url:port
    api_key="api_key" # here replace with your api key
    
    dataiku.set_remote_dss(host, api_key)
    dataiku.set_default_project_key("promect_name") # shere replace with the project you want to create a dataset in
    
    # Example: load a DSS dataset as a Pandas dataframe
    mydataset = dataiku.Dataset("mydataset_name")
    mydataset_df = mydataset.get_dataframe()

    However, please note generally, the API node shouldn't be used to load large datasets and/or perform large computations.

    Best,

    Vitaliy

Answers

  • VitaliyD
    VitaliyD Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer Posts: 102 Dataiker

    Hi,

    No, it is not possible to return multiple rows with a lookup endpoint. You will need to use a custom python endpoint for that.

    Best,

    Vitaliy

  • MoseleyDanielCr
    MoseleyDanielCr Registered Posts: 2

    Do you know if I can reference a dataset in a python endpoint? I tried it in the standard way and it didn't work

    i.e.

    import dataiku

    dataset = dataiku.Dataset('dataset_name')

    df = dataset.get_dataframe()

Setup Info
    Tags
      Help me…