API Nodes CICD, value of these variables  'apinodes' and 'url'

shajay12
shajay12 Registered Posts: 14 ✭✭✭
edited July 16 in Using Dataiku

Hi Everyone,

Please once check this. In this Article Building a Jenkins pipeline for API services in Dataiku DSS

what is the value of these variables 'apinodes' and 'url' .....code pasted below?

def build_apinode_client(params):
    client_design = dataikuapi.DSSClient(params["host"], params["api"])
    api_deployer = client_design.get_apideployer()
    api_url = api_deployer.get_infra(params["api_dev_infra_id"]).get_settings().get_raw()['apiNodes'][0]['url']
    return dataikuapi.APINodeClient(api_url, params["api_service_id"])

Answers

  • Sarina
    Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 317 Dataiker
    edited July 17

    Hi @shajay12
    ,

    This code snippet pulls the URL that is defined for the relevant API infrastructure defined under Deployer > API services > Infrastructures > the infrastructure that corresponds to params["api_dev_infra_id"] > Settings > API Nodes: 

    Screen Shot 2021-12-21 at 3.00.08 PM.png

    The code will return the URL of the first api node in this infrastructure, so in the example above, will return "http://localhost:19444".

    It might help to break down the python snippet and run the "outer" query to see what exactly gets returned at each step. For example, you can run the following to see all of the raw output that gets returned, before narrowing down to "apiNode" and the relevant "url":

    api_deployer.get_infra(params["api_dev_infra_id").get_settings().get_raw()


    This will return the full dictionary of settings for your API node infrastructure. For me, you can see the results below. apiNodes is a an array of any API nodes that you have listed in your infrastructure. In my screenshot above, you'll see I have two listed. For the sample code, you get the first apiNode object (apiNodes[0]), and then you get the url element from that object. In this case, that would return "http://localhost:19444":

    Screen Shot 2021-12-21 at 3.02.52 PM.png

    To sum up, "apiNodes" and "url" are simply dictionary keys, that you can use to pull the relevant api node and url information from your API infrastructure object.

    I hope that information is helpful, please let me know if you have any questions about this!

    Thank you,
    Sarina

  • shajay12
    shajay12 Registered Posts: 14 ✭✭✭

    Hi Sarina,

    I have Kubernetes-based infrastructure in the API deployer. So I do not have API Nodes installed.

    I get this error when I run the above code.

    Docs: https://docs.pytest.org/en/stable/warnings.html
    -------- generated xml file: /azp/agent/_work/2/s/reports/DEV_TEST.xml ---------
    =========================== short test summary info ============================
    FAILED 2_deploy_dev/test_dev.py::test_standard_call - KeyError: 'apiNodes'
    FAILED 2_deploy_dev/test_dev.py::test_missing_param - KeyError: 'apiNodes'
    ======================== 2 failed, 2 warnings in 0.68s =========================
    ##[error]Bash exited with code '1'.
    Async Command Start: Publish test results
    Publishing test results to test run '8096'.
    TestResults To Publish 2, Test run id:8096
    Test results publishing 2, remaining: 0. Test run id: 8096
    Published Test Run: https://dev.azure.com/fxe-data-mgmt/Controller%20Tier/_TestManagement/Runs?runId=8096&_a=runCharts
    Flaky failed test results are opted out of pass percentage
    Async Command End: Publish test results
    Finishing: pytest

    Here is the code. ::::::

    import dataikuapi

    def build_apinode_client(params):

    client_design = dataikuapi.DSSClient(params["host"], params["api"])

    client_design._session.verify = False

    api_deployer = client_design.get_apideployer()

    api_url = api_deployer.get_infra(params["api_dev_infra_id"]).get_settings().get_raw()['apiNodes'][0]['url']

    return dataikuapi.APINodeClient(api_url, params["api_service_id"])

    def test_standard_call(params):

    client = build_apinode_client(params)

    print("Test is using API node URL {}".format(client.base_uri))

    record_to_predict = {

    "XXXX": "XXXX",

    "XXXXXXXXXX": XXXXXX,

    "XXXXXXXX": XXXXX,

    "XXXX": "XXXXXXXX",

    "XXXX": FXXXe,

    "XXXXXXXX": XXXXX,

    "XXXXXXXX": XXXXXXX,

    "XXXXX": XXXXXXX,

    "XXXXXXXX": XXXXX,

    "XXXXXXX": XXXXXXX,

    "XXXXXXX": XXXXXX,

    "XXXXXXXXXXX": XXXX,

    "XXXXXXX": XXXXX,

    "XXXXXXXXXXX": XXXXXXXXX,

    "XXXXXXXX": XXXX,

    "XXXXXXXXXXXXXXXXX": XXXXXXXXX,

    "XXXXXXXXXX": 10,

    "XXXXXXXXXX": XXXXXX,

    "XXXXXX": 3,

    "XXXXXXX": XX,

    "XXXXXXXX": XXXXX,

    "XXXXXXXX": 1,

    "XXXXXXXXX": "cluster_4"

    }

    prediction = client.predict_record(params["api_endpoint_id"], record_to_predict)

    assert prediction['result']['prediction'] == '1', "Prediction should be 1 but is {}".format(prediction['result']['prediction'])

  • Sarina
    Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 317 Dataiker
    edited July 17

    Hi @shajay12
    ,

    Thank you for the clarification! Indeed for a K8s infrastructure, there is no `apiNodes` field associated with the infrastructure, so the specified piece of code will not apply. You get the key error because this field is not part of the infrastructure object. The endpoint URL will instead be part of however your K8s deployment service exposition is set.

    The sample code provided in the knowledge base article simply creates an APINodeClient object for you, so in order to mimic it for a K8s infrastructure you can identify how your services are exposed.

    For a K8s deployment, I suggest taking a look at the "Sample code" Python for your API deployment. This will provide a sample of how you would create an APINodeClient for your specific deployment. Here's an example of a K8s deployment APINodeClient creation where my services exposition is set to "LoadBalancer":

    Screen Shot 2021-12-25 at 3.23.17 PM.png

    You can pull the relevant "External-IP" for the service on your K8s cluster with something like this:

    $ kubectl get services
    NAME                                                          TYPE           CLUSTER-IP      EXTERNAL-IP                                                              PORT(S)           AGE
    dku-mad-service-python-service-on-kubernetes-infrastructure   LoadBalancer   172.20.242.27   a00742892b5034cc09e2c44dba1703ef-358364300.us-east-2.elb.amazonaws.com   12000:31328/TCP   3m36s
    


    The EXTERNAL-IP field will provide your APINodeClient URI parameter.

    Thanks,
    Sarina

  • varunsampya
    varunsampya Registered Posts: 1

    Could you please give me the python code to deploy the service endpoints into kubernetes infrastructure.

Setup Info
    Tags
      Help me…