Containerized Jobs from automation nodes give connection refused errors

DrissiReda

I'm running a design and an automation node.

I execute all of my recipe workloads in a kubernetes cluster.

When I launch a project's visual recipe from the design node. It works well, the job is launched in kubernetes and is successful.

Does this mean I need to open more ports on the Design node? Is there any way to overwrite the port range?
What limitations would I face if I limit it to 10 or less ports?

When I launch the same visual recipe from the same project from the automation node I get the following error:

Connect to dataiku-0.dataiku.processing.svc.cluster.local:35543 [dataiku-0.dataiku.processing.svc.cluster.local/10.244.235.148] failed: Connection refused (Connection refused), caused by: ConnectException: Connection refused (Connection refused)

The configuration is the same between the automation and design nodes.

The whole output is attached.

Sarina

Hi @DrissiReda
,

Sorry for the delay! It is the case that all ports are expected to be open between your cluster and the DSS backend: https://doc.dataiku.com/dss/latest/containers/setup-k8s.html#docker-and-kubectl-setup. Note that the container "test" will only test on a single port, while regular jobs can run on any port.

If there is a problem connecting to the DSS backend host, you may also need to set the following setting to specify your DSS backend IP address:
https://doc.dataiku.com/dss/latest/containers/troubleshooting.html#requests-exceptions-connectionerror

Hopefully all is working for you at this point, let us know if you have any other questions about this.

Thanks,
Sarina