API Nodes CICD, value of these variables 'apinodes' and 'url'

shajay12

Hi Everyone,

Please once check this. In this Article Building a Jenkins pipeline for API services in Dataiku DSS

what is the value of these variables 'apinodes' and 'url' .....code pasted below?

def build_apinode_client(params):
    client_design = dataikuapi.DSSClient(params["host"], params["api"])
    api_deployer = client_design.get_apideployer()
    api_url = api_deployer.get_infra(params["api_dev_infra_id"]).get_settings().get_raw()['apiNodes'][0]['url']
    return dataikuapi.APINodeClient(api_url, params["api_service_id"])

Sarina

Hi @shajay12
,

This code snippet pulls the URL that is defined for the relevant API infrastructure defined under Deployer > API services > Infrastructures > the infrastructure that corresponds to params["api_dev_infra_id"] > Settings > API Nodes: 

The code will return the URL of the first api node in this infrastructure, so in the example above, will return "http://localhost:19444".

It might help to break down the python snippet and run the "outer" query to see what exactly gets returned at each step. For example, you can run the following to see all of the raw output that gets returned, before narrowing down to "apiNode" and the relevant "url":

api_deployer.get_infra(params["api_dev_infra_id").get_settings().get_raw()

This will return the full dictionary of settings for your API node infrastructure. For me, you can see the results below. apiNodes is a an array of any API nodes that you have listed in your infrastructure. In my screenshot above, you'll see I have two listed. For the sample code, you get the first apiNode object (apiNodes[0]), and then you get the url element from that object. In this case, that would return "http://localhost:19444":

To sum up, "apiNodes" and "url" are simply dictionary keys, that you can use to pull the relevant api node and url information from your API infrastructure object.

I hope that information is helpful, please let me know if you have any questions about this!

Thank you,
Sarina

shajay12

Hi Sarina,

I have Kubernetes-based infrastructure in the API deployer. So I do not have API Nodes installed.

I get this error when I run the above code.

Docs: https://docs.pytest.org/en/stable/warnings.html
-------- generated xml file: /azp/agent/_work/2/s/reports/DEV_TEST.xml ---------
=========================== short test summary info ============================
FAILED 2_deploy_dev/test_dev.py::test_standard_call - KeyError: 'apiNodes'
FAILED 2_deploy_dev/test_dev.py::test_missing_param - KeyError: 'apiNodes'
======================== 2 failed, 2 warnings in 0.68s =========================
##[error]Bash exited with code '1'.
Async Command Start: Publish test results
Publishing test results to test run '8096'.
TestResults To Publish 2, Test run id:8096
Test results publishing 2, remaining: 0. Test run id: 8096
Published Test Run: https://dev.azure.com/fxe-data-mgmt/Controller%20Tier/_TestManagement/Runs?runId=8096&_a=runCharts
Flaky failed test results are opted out of pass percentage
Async Command End: Publish test results
Finishing: pytest

Here is the code. ::::::

import dataikuapi

def build_apinode_client(params):

client_design = dataikuapi.DSSClient(params["host"], params["api"])

client_design._session.verify = False

api_deployer = client_design.get_apideployer()

api_url = api_deployer.get_infra(params["api_dev_infra_id"]).get_settings().get_raw()['apiNodes'][0]['url']

return dataikuapi.APINodeClient(api_url, params["api_service_id"])

def test_standard_call(params):

client = build_apinode_client(params)

print("Test is using API node URL {}".format(client.base_uri))

record_to_predict = {

"XXXX": "XXXX",

"XXXXXXXXXX": XXXXXX,

"XXXXXXXX": XXXXX,

"XXXX": "XXXXXXXX",

"XXXX": FXXXe,

"XXXXXXXX": XXXXX,

"XXXXXXXX": XXXXXXX,

"XXXXX": XXXXXXX,

"XXXXXXXX": XXXXX,

"XXXXXXX": XXXXXXX,

"XXXXXXX": XXXXXX,

"XXXXXXXXXXX": XXXX,

"XXXXXXX": XXXXX,

"XXXXXXXXXXX": XXXXXXXXX,

"XXXXXXXX": XXXX,

"XXXXXXXXXXXXXXXXX": XXXXXXXXX,

"XXXXXXXXXX": 10,

"XXXXXXXXXX": XXXXXX,

"XXXXXX": 3,

"XXXXXXX": XX,

"XXXXXXXX": XXXXX,

"XXXXXXXX": 1,

"XXXXXXXXX": "cluster_4"

}

prediction = client.predict_record(params["api_endpoint_id"], record_to_predict)

assert prediction['result']['prediction'] == '1', "Prediction should be 1 but is {}".format(prediction['result']['prediction'])

Sarina

Hi @shajay12
,

Thank you for the clarification! Indeed for a K8s infrastructure, there is no `apiNodes` field associated with the infrastructure, so the specified piece of code will not apply. You get the key error because this field is not part of the infrastructure object. The endpoint URL will instead be part of however your K8s deployment service exposition is set.

The sample code provided in the knowledge base article simply creates an APINodeClient object for you, so in order to mimic it for a K8s infrastructure you can identify how your services are exposed.

For a K8s deployment, I suggest taking a look at the "Sample code" Python for your API deployment. This will provide a sample of how you would create an APINodeClient for your specific deployment. Here's an example of a K8s deployment APINodeClient creation where my services exposition is set to "LoadBalancer":

You can pull the relevant "External-IP" for the service on your K8s cluster with something like this:

$ kubectl get services
NAME                                                          TYPE           CLUSTER-IP      EXTERNAL-IP                                                              PORT(S)           AGE
dku-mad-service-python-service-on-kubernetes-infrastructure   LoadBalancer   172.20.242.27   a00742892b5034cc09e2c44dba1703ef-358364300.us-east-2.elb.amazonaws.com   12000:31328/TCP   3m36s

 
The EXTERNAL-IP field will provide your APINodeClient URI parameter.

Thanks,
Sarina

varunsampya

Could you please give me the python code to deploy the service endpoints into kubernetes infrastructure.