Discover all of the brand-new features and improvements to existing capabilities in the Dataiku 11.3 updateLET'S GO

Containerized Jobs from automation nodes give connection refused errors

DrissiReda
Level 4
Containerized Jobs from automation nodes give connection refused errors

I'm running a design and an automation node.

 

I execute all of my recipe workloads in a kubernetes cluster.

 

When I launch a project's visual recipe from the design node. It works well, the job is launched in kubernetes and is successful.


Does this mean I need to open more ports on the Design node? Is there any way to overwrite the port range?
What limitations would I face if I limit it to 10 or less ports?

When I launch the same visual recipe from the same project from the automation node I get the following error:

 

 

 

Connect to dataiku-0.dataiku.processing.svc.cluster.local:35543 [dataiku-0.dataiku.processing.svc.cluster.local/10.244.235.148] failed: Connection refused (Connection refused), caused by: ConnectException: Connection refused (Connection refused)

 

 

 

 

The configuration is the same between the automation and design nodes.

The whole output is attached.

 

 

0 Kudos
1 Reply
SarinaS
Dataiker

Hi @DrissiReda,

Sorry for the delay! It is the case that all ports are expected to be open between your cluster and the DSS backend: https://doc.dataiku.com/dss/latest/containers/setup-k8s.html#docker-and-kubectl-setup. Note that the container "test" will only test on a single port, while regular jobs can run on any port. 

If there is a problem connecting to the DSS backend host, you may also need to set the following setting to specify your DSS backend IP address: 
https://doc.dataiku.com/dss/latest/containers/troubleshooting.html#requests-exceptions-connectionerr... 

Hopefully all is working for you at this point, let us know if you have any other questions about this. 

Thanks,
Sarina

0 Kudos