You now have until September 15th to submit your use case or success story to the 2022 Dataiku Frontrunner Awards!ENTER YOUR SUBMISSION

Containerized Jobs from automation nodes give connection refused errors

DrissiReda
Level 3
Containerized Jobs from automation nodes give connection refused errors

I'm running a design and an automation node.

 

I execute all of my recipe workloads in a kubernetes cluster.

 

When I launch a project's visual recipe from the design node. It works well, the job is launched in kubernetes and is successful.


Does this mean I need to open more ports on the Design node? Is there any way to overwrite the port range?
What limitations would I face if I limit it to 10 or less ports?

When I launch the same visual recipe from the same project from the automation node I get the following error:

 

 

 

Connect to dataiku-0.dataiku.processing.svc.cluster.local:35543 [dataiku-0.dataiku.processing.svc.cluster.local/10.244.235.148] failed: Connection refused (Connection refused), caused by: ConnectException: Connection refused (Connection refused)

 

 

 

 

The configuration is the same between the automation and design nodes.

The whole output is attached.

 

 

0 Kudos
1 Reply
SarinaS
Dataiker
Dataiker

Hi @DrissiReda,

Sorry for the delay! It is the case that all ports are expected to be open between your cluster and the DSS backend: https://doc.dataiku.com/dss/latest/containers/setup-k8s.html#docker-and-kubectl-setup. Note that the container "test" will only test on a single port, while regular jobs can run on any port. 

If there is a problem connecting to the DSS backend host, you may also need to set the following setting to specify your DSS backend IP address: 
https://doc.dataiku.com/dss/latest/containers/troubleshooting.html#requests-exceptions-connectionerr... 

Hopefully all is working for you at this point, let us know if you have any other questions about this. 

Thanks,
Sarina

0 Kudos