Python recipe taking a lot of time to complete

yjagger · March 2023

hi,

I am trying to run a python recipe using 'DSS builtin env', and it is taking a lot of time for it to complete.

From the logs it looks like that it takes a lot of time to create / establish connection, although I am not sure if the problem is something related to no vacant connection available in the pool.

In the logs i can see the line 'running use code' and then it hangs for about 5 minutes and prints the lines - 'Verifying SSL calls with cert....' (as below)

[2023/03/07-06:16:28.498] [kubectl-stream-logs-out-151] [INFO] [dku.utils]  - [2023-03-07 06:16:27,576] [1/MainThread] [INFO] [root] Running user code
[2023/03/07-06:16:28.499] [kubectl-stream-logs-out-151] [INFO] [dku.utils]  - [2023-03-07 06:16:28,491] [24/MainThread] [DEBUG] [root] Verifying SSL calls with certificate /home/dataiku/rpc_server_cert.pem
[2023/03/07-06:16:28.499] [kubectl-stream-logs-out-151] [INFO] [dku.utils]  - [2023-03-07 06:16:28,491] [24/MainThread] [DEBUG] [root] Verifying SSL calls with certificate /home/dataiku/rpc_server_cert.pem
[2023/03/07-06:16:28.499] [kubectl-stream-logs-out-151] [INFO] [dku.utils]  - [2023-03-07 06:16:28,492] [24/MainThread] [DEBUG] [root] Verifying SSL calls with certificate /home/dataiku/rpc_server_cert.pem
[2023/03/07-06:16:28.499] [kubectl-stream-logs-out-151] [INFO] [dku.utils]  - [2023-03-07 06:16:28,492] [24/MainThread] [DEBUG] [root] Verifying SSL calls with certificate /home/dataiku/rpc_server_cert.pem
[2023/03/07-06:16:28.529] [kubectl-stream-logs-out-151] [INFO] [dku.utils]  - [2023-03-07 06:16:28,495] [24/MainThread] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): pdue2-diku-vm02.amd.com:43215

Can you describe what may be happening here ?

Operating system used: Windows

Alexandru · March 2023

Hi,
From what I see there spending a lot of time running the code itself. It looks like within the code you are making connections to pdue2-diku-vm02.amd.com which may be taking a long. The best way to understand where time is spent would be to add additional logging within your code by instrumenting with print statements after each part of your code or within loops.

If you need further assistance please open a support ticket and share job diagnostics with your instrumented code. ( not share the job diagnostics on community)

https://doc.dataiku.com/dss/latest/troubleshooting/problems/job-fails.html#getting-a-job-diagnosis

Thanks

Python recipe taking a lot of time to complete

Answers

Categories

Setup Info

Tags