Jenkins - Project and API

Jaspal

Hello,

I'm seeking documentation/guidance on implementing a Jenkins pipeline for our specific use case, but I haven't found any helpful resources. Our API service consists of two endpoints (Python Function Endpoint and Model Predict Endpoint), and the Project Library containing business logic sits between them. The trained model resides on the automation node.

As per my understanding, we need to follow these steps for deployment:

1. Bundle the project and move it to the Test Automation Node.
2. Create a new version of the API and deploy it to the Test API node.
3. Follow the same sequence for moving to production.

Given this scenario, we need guidance on designing our Jenkins pipeline:

Scenario 1:
Pipeline 1: Bundle the project and move it to the Test Automation Node.
Pipeline 2: Create a new API version and deploy it to the Test API node.
Pipeline 3: Bundle the project and move it to the Production Automation Node.
Pipeline 4: Create a new API version and deploy it to the Production API node.

Scenario 2:
Pipeline 1: Bundle the project, move it to the Test Automation Node, create a new API version, and deploy it to the Test API node.
Pipeline 2: Bundle the project, move it to the Production Automation Node, create a new API version, and deploy it to the Production API node.

Scenario 3:
Pipeline 1: Bundle the project and move it to the Test Automation Node, create a new API version and deploy it to the Test API node, bundle the project and move it to the Production Automation Node, and create a new API version and deploy it to the Production API node.

Please advise on which scenario would be the most suitable for our Jenkins pipeline implementation and if there's any relevant documentation available?

Turribeach

The following documentation pages will help you. Regarding your scenarios it really depends on how you do changes and how is your API dependant on your project. My choice will be Scenario 1 because each stage is a different pipeline and you can release API and project separately as you will not always have changes that affect both at the same time.

https://knowledge.dataiku.com/latest/mlops-o16n/ci-cd/tutorial-getting-started-ci-cd.html

https://knowledge.dataiku.com/latest/mlops-o16n/ci-cd/tutorial-jenkins-pipeline.html

https://knowledge.dataiku.com/latest/mlops-o16n/ci-cd/tutorial-jenkins-pipeline-project-deployer.html

https://knowledge.dataiku.com/latest/mlops-o16n/ci-cd/tutorial-jenkins-pipeline-api-services.html