Artifactory Versioning for Bundle

sj0071992

Hi Team,

I am following the below link for implementing CI/CD in Dataiku.

https://knowledge.dataiku.com/latest/kb/o16n/ci-cd/index.html#what-s-next

It is mentioned here like Artifactory is used to store the Bundle.

I just want to confirm how we can maintain the version of the Bundle and roll it back when required (to PREPROD/PROD).

Thanks in Advance.

fsergot

Hello,

In this sample, Artifactory is used for archiving. The actual bundles that are deployed to automation nodes are stored within the DSS node hosting the Deployer (the Design node in this case).

As for the rollback, there are many options:

1. When a bundle is uploaded and activated on the Automation node, it stays there unless you manually delete it. Meaning if you put a new version and want to rollback, you just need to reactivate the previous running bundle. That is actually what is one in the sample project (the last part in deploy_bundle.py ). That's the simplest and recommended approach.
2. If you delete the bundles on the Automation node, you can still redeploy the previous bundle from the Deployer as all bundles deployed are stored there. In this case, the complex part is to know which bundle to restore.
3. If you really want to source from Artifactory, you need to change the deploy code so that you take the bundle zip from Artifactory and publish it manually to Deployer (using the DSSProjectDeployer.upload_bundle API. And then deploy this bundle. More complex to do.

Hope this helps

sj0071992

Hi,

This is really great.

But how can I do a rollback to the previous bundle based on my test results which I will run on TEST Node? I mean to say what if something is not working properly after deployment to TEST (test results failed) then how can I do a rollback in that case?

Thanks in Advance

fsergot

Hello,

In a pure CI/CD logic, all versions that pass tests should be moved to prod and if any issue occur, the rollback to the previous version should be enough to get the last working bundle.

If, for any reason, you are not certain of this logic or want to base the rollback on the test result status, there is nothing out of the box for that, so you need to code it.

As a guideline and personal thoughts, I can suggest 2 ideas:

1. Storing the test result status within Artifactory (such as with Property Sets ). If need be, you can then query Artifactory for the last release that had a successful test and use the associated bundle
2. You can store the bundleID used as a jenkins build property. In case of failure, you then need to fetch back the last successful run in Jenkins, extract this property that gives you the bundle ID associated to it and deploy this particular bundle ID with DSS deployer (to me, this is the best option)

Note that those are 2 ideas, I have not tested them.

Hope this helps

You can use the same logic as in deploy_bundle.py from the deploy to prod stage, this is exactly what is done there:

1. Before deploying the new bundle, keep the currently deployed bundle ID
2. Deploy the new bundle
3. Run the test
4. If there is a failure, redeploy the bundle ID you save at step 1

With this, you always have the last working bundle on your TEST node (and you haven't touch the preprod/prod

sj0071992

Hi,

Thanks for the Solution. I was just looking into the link but could you please tell me about the arguments below, where can I find these values in the code deploy_bundle.py

Specially infra = sys.argv[5].

import dataikuapi
import sys

pdpl_host = sys.argv[1]
pdpl_apiKey = sys.argv[2]
project = sys.argv[3]
bundle_id = sys.argv[4]
infra = sys.argv[5]
auto_host = sys.argv[6]
auto_apiKey = sys.argv[7]

fsergot

Hello,

Those are command-line arguments setup within Jenkins groovy pipeline file (sample here)

sh "python 4_deploy_prod/deploy_bundle.py '${DESIGN_URL}' '${DESIGN_API_KEY}' '${DSS_PROJECT}' '${bundle_name}' '${AUTO_PROD_ID}' ${AUTO_PROD_URL} ${AUTO_PROD_API_KEY}"

Those variables here are the one defined within Jenkins project.

In your case, sys.argv[5] is the fifth argument, which is '${AUTO_PROD_ID}'. In the example, this is the infrastructure ID as defined in DSS Project Deployer (as you can see in the main doc: auto-prod in the sample case)