Best practices for real-time APIs?
Has anyone got any best practices they can recommend for development, testing and deploying real-time API services? At the moment we have considered documenting and testing under load. Does the community know of anything else we can add?
Answers
-
Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 412 NeuronHi @Charlie.
I think it will depend on your Dataiku setup. Are you using a remote deployer, with an automation node for testing and another for deployment? Or are you using the local deployer to your DSS instance?
-
Hi @Ignacio_Toledo, thanks for coming back to me. We are using a remote deployer and we have two automation nodes, one for testing and one for production.
-
Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 412 Neuron
Hi @Charlie. I appreciate your patience with this, I didn't get a notification by email that you have answered already.
So, that is a similar setup to the one we have at my organization, except instead of a testing node, we have a "pre-production" node. The initial testing is done in the design (development) instance, and then we deploy in pre-production to test the integration with other projects when there are dependences, and see how it reacts in real-time, but not yet in production.
I realize the topic is broad. Is there a specific area you would like me to describe more in detail? Or just the general process is all that you need?
-
Thanks @Ignacio_Toledo. It would be great to hear about the general process and any standards you implemented. E.g., standards for Python endpoints, monitoring and alerting, and documentation.