Hello everyone!
I configured a CI/CD pipeline based on this reference and it was working fine:
- Creating a bundle
- Publishing the bundle to Deployer
- Deploying to Automation Dev node
- ...
OBS #1: In order to do these steps, the pipeline code uses `dataikuapi` Python library, connecting to our Design Node through its host, port, and global key.
OBS #2: The global key has full permissions and hasn't changed for months.
OBS #3: The connection is successful, as the code can effectively list projects, find the project and execute steps 1 and 2.
OBS #4: If I create a notebook in my Design node and use `dataiku` library (instead of dataikuapi) to initiate the local client (without the need of setting host, port, and global key), then these three steps of my pipeline work smoothly with not issues.
OBS #5: If I create a notebook on a different VM (simulating the Azure DevOps pipeline) and use `dataikuapi` to connect to the Design Node (setting host, port, and global key), steps 1 and 2 work fine (in theory eliminating network issues and validating the global key), but step three breaks with this tricky error ERR_PROJECT_DEPLOYER_AUTOMATION_NODE_REMOTE_ERROR.
OBS #6: I tested recreating the global keys, using different projects, etc., and still got the same error.
OBS #7: As far as we could check, there was no change in the pipeline or our infrastructure or our VMs or our DSS installation or the global keys. This is what most intrigues us.
The code is as follows:
import dataiku
import dataikuapi
### Connecting
# !!! IF I USE THIS, IT BREAKS AT THE END
client = dataikuapi.DSSClient("http://our_ip:post", "********")
# !!! IF I USE THIS, FROM A NOTEBOOK IN MY DESIGN NODE, IT WORKS TIL THE END
# client = dataiku.api_client()
project_id = "MY_PROJECT_ID"
bundle_id = "bundle_2022-01-28_00-12-15"
# Create and publish bundle
project = client.get_project(project_id)
project.export_bundle(bundle_id)
project.publish_bundle(bundle_id)
# Creates deployment and deploys
project_id in [p["projectKey"] for p in client.list_projects()] # Returns True as expected
pdpl = client.get_projectdeployer()
infra = "dev-automation-node"
project_id in [p.id for p in pdpl.list_projects()] # Returns False as expected
pdpl_proj = pdpl.get_project(project_id)
deployments = pdpl_proj.get_status().get_deployments(infra)
len(deployments) > 0 # Returns False as expected
dp_id = pdpl_proj.id + "-on-" + infra
deployment = pdpl.create_deployment(dp_id, pdpl_proj.id, infra, bundle_id)
update_exec = deployment.start_update()
update_exec.wait_for_result() # Here I get the ERR_PROJECT_DEPLOYER_AUTOMATION_NODE_REMOTE_ERROR
Operating system used: Linux
