Submit your inspiring success story or innovative use case to the 2022 Dataiku Frontrunner Awards! ENTER YOUR SUBMISSION

Python dataikuapi raising ERR_PROJECT_DEPLOYER_AUTOMATION_NODE_REMOTE_ERROR

Solved!
daniel_adornes
Level 3
Python dataikuapi raising ERR_PROJECT_DEPLOYER_AUTOMATION_NODE_REMOTE_ERROR

Hello everyone!

I configured a CI/CD pipeline based on this reference and it was working fine:

  1. Creating a bundle
  2. Publishing the bundle to Deployer
  3. Deploying to Automation Dev node
  4. ...

OBS #1: In order to do these steps, the pipeline code uses `dataikuapi` Python library, connecting to our Design Node through its host, port, and global key. 

OBS #2: The global key has full permissions and hasn't changed for months.

OBS #3: The connection is successful, as the code can effectively list projects, find the project and execute steps 1 and 2.

OBS #4: If I create a notebook in my Design node and use `dataiku` library (instead of dataikuapi) to initiate the local client (without the need of setting host, port, and global key), then these three steps of my pipeline work smoothly with not issues.

OBS #5: If I create a notebook on a different VM (simulating the Azure DevOps pipeline) and use `dataikuapi` to connect to the Design Node (setting host, port, and global key), steps 1 and 2 work fine (in theory eliminating network issues and validating the global key), but step three breaks with this tricky error ERR_PROJECT_DEPLOYER_AUTOMATION_NODE_REMOTE_ERROR.

OBS #6: I tested recreating the global keys, using different projects, etc., and still got the same error.

OBS #7: As far as we could check, there was no change in the pipeline or our infrastructure or our VMs or our DSS installation or the global keys. This is what most intrigues us.

The code is as follows:

 

import dataiku
import dataikuapi

### Connecting 

# !!! IF I USE THIS, IT BREAKS AT THE END
client = dataikuapi.DSSClient("http://our_ip:post", "********")

# !!! IF I USE THIS, FROM A NOTEBOOK IN MY DESIGN NODE, IT WORKS TIL THE END
# client = dataiku.api_client()

project_id = "MY_PROJECT_ID"

bundle_id = "bundle_2022-01-28_00-12-15"

# Create and publish bundle

project = client.get_project(project_id)

project.export_bundle(bundle_id)

project.publish_bundle(bundle_id)

# Creates deployment and deploys

project_id in [p["projectKey"] for p in client.list_projects()] # Returns True as expected

pdpl = client.get_projectdeployer()

infra = "dev-automation-node"

project_id in [p.id for p in pdpl.list_projects()] # Returns False as expected

pdpl_proj = pdpl.get_project(project_id)

deployments = pdpl_proj.get_status().get_deployments(infra)

len(deployments) > 0 # Returns False as expected

dp_id = pdpl_proj.id + "-on-" + infra

deployment = pdpl.create_deployment(dp_id, pdpl_proj.id, infra, bundle_id)

update_exec = deployment.start_update()

update_exec.wait_for_result() # Here I get the ERR_PROJECT_DEPLOYER_AUTOMATION_NODE_REMOTE_ERROR

 


Operating system used: Linux

0 Kudos
1 Solution
SarinaS
Dataiker
Dataiker

Hi @daniel_adornes,

Thank you for your setup details and clear description of the issue you are encountering when you see receive the error message "ERR_PROJECT_DEPLOYER_AUTOMATION_NODE_REMOTE_ERROR".

It looks like you are currently using a global API key in your code definition. Instead, please use a personal API key, as there are some actions that require a personal API key. 

Let me know if you run into any issues after switching to a personal API key. 

Thanks,
Sarina 

View solution in original post

2 Replies
SarinaS
Dataiker
Dataiker

Hi @daniel_adornes,

Thank you for your setup details and clear description of the issue you are encountering when you see receive the error message "ERR_PROJECT_DEPLOYER_AUTOMATION_NODE_REMOTE_ERROR".

It looks like you are currently using a global API key in your code definition. Instead, please use a personal API key, as there are some actions that require a personal API key. 

Let me know if you run into any issues after switching to a personal API key. 

Thanks,
Sarina 

daniel_adornes
Level 3
Author

@SarinaS It worked with a personal key as you suggested!! Thank you very much!!

Still, I'm almost sure that it was working before with my global key, which still intrigues me.

Well, thank you and have a great weekend!