Error compute kernel failure new Statistics Feature

Jailbreaker
Jailbreaker Registered Posts: 9 ✭✭✭✭
edited July 2024 in Setup & Configuration

Hello,

I updated our Dataiku Plateform from 5.X to the 8. version.

So when we try to use the new Statistics feature on a Dataset, I get the following error:

Unexpected EDA kernel failure

com.dataiku.dip.exceptions.CodedException: EDA docker kernel failed to start, caused by: CodedException: EDA docker kernel failed to start, caused by: SecretKernelTimeoutException: Subprocess failed to connect, it probably crashed at startup. Check the logs., caused by: SocketException: Socket closed
Here are the logs I got from the backend.log file

[2020/11/25-19:17:01.508] [qtp1528769018-365630] [DEBUG] [dku.tracing]  - [ct: 1] Start call: /api/futures/get-update [GET] user=admin [futureId=1EMRVvBF]
[2020/11/25-19:17:01.508] [qtp1528769018-365630] [DEBUG] [dku.tracing]  - [ct: 1] Done call: /api/futures/get-update [GET] time=1ms user=admin [futureId=1EMRVvBF] thrown=ExecutionException
[2020/11/25-19:17:01.508] [qtp1528769018-365630] [ERROR] [dip.controllers]  - API call '/dip/api/futures/get-update' failed
java.util.concurrent.ExecutionException: com.dataiku.dip.exceptions.CodedException: EDA local kernel failed to start
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at com.dataiku.dip.eda.compute.engine.LazyRunner.getOrBuildRunner(LazyRunner.java:88)
        at com.dataiku.dip.eda.compute.engine.LazyRunner.compute(LazyRunner.java:38)
        at com.dataiku.dip.eda.compute.engine.SmoothlyClosableRunner.compute(SmoothlyClosableRunner.java:23)
        at com.dataiku.dip.eda.compute.caching.CachedRunner.compute(CachedRunner.java:83)
        at com.dataiku.dip.eda.worksheets.CardComputeService.computeResultsWithRunner(CardComputeService.java:78)
        at com.dataiku.dip.eda.worksheets.CardComputeService.computeResults_NT(CardComputeService.java:67)
        at com.dataiku.dip.eda.worksheets.CardComputeService$1.compute(CardComputeService.java:50)
        at com.dataiku.dip.eda.worksheets.CardComputeService$1.compute(CardComputeService.java:35)
        at com.dataiku.dip.futures.SimpleFutureThread.execute(SimpleFutureThread.java:36)
        at com.dataiku.dip.futures.FutureThreadBase.run(FutureThreadBase.java:88)
Caused by: com.dataiku.dip.exceptions.CodedException: EDA local kernel failed to start
        at com.dataiku.dip.eda.compute.engine.python.LocalPythonKernel.start(LocalPythonKernel.java:33)
        at com.dataiku.dip.eda.compute.ComputeService.createSampleKernel(ComputeService.java:146)
        at com.dataiku.dip.eda.compute.ComputeService.access$000(ComputeService.java:52)
        at com.dataiku.dip.eda.compute.ComputeService$1.call(ComputeService.java:93)
        at com.dataiku.dip.eda.compute.ComputeService$1.call(ComputeService.java:87)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: com.dataiku.dip.io.SocketBlockLink$SecretKernelTimeoutException: Subprocess failed to connect, it probably crashed at startup. Check the logs.
        at com.dataiku.dip.io.SocketBlockLink.waitForConnection(SocketBlockLink.java:70)
        at com.dataiku.dip.io.SecretProtectedKernelLink.waitForProcess(SecretProtectedKernelLink.java:39)
        at com.dataiku.dip.io.SecretProtectedKernelLink.waitForProcess(SecretProtectedKernelLink.java:48)
        at com.dataiku.dip.io.PythonSecretProtectedKernel.start(PythonSecretProtectedKernel.java:108)
        at com.dataiku.dip.eda.compute.engine.python.LocalPythonKernel.start(LocalPythonKernel.java:29)
        ... 8 more
Caused by: java.net.SocketException: Socket closed
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
        at java.net.ServerSocket.implAccept(ServerSocket.java:545)
        at java.net.ServerSocket.accept(ServerSocket.java:513)
        at com.dataiku.dip.io.SocketBlockLink.waitForConnection(SocketBlockLink.java:66)
        ... 12 more
[2020/11/25-19:17:01.509] [qtp1528769018-365630] [INFO] [dip.controllers]  - Err for /dip/api/futures/get-update

· Share on FacebookShare on Twitter

Best Answer

Answers

  • AdrienL
    AdrienL Dataiker, Alpha Tester Posts: 196 Dataiker

    Attached backend logs seem to show that the sortedcontainer package is missing. You can try rebuilding your python built-in environment.

    · Share on FacebookShare on Twitter
  • Jailbreaker
    Jailbreaker Registered Posts: 9 ✭✭✭✭

    I ran the rebuilt command and added this librairy, restart my DSS Instance but still the same error.

    Attached a screenshot our installed packages in the python_env

    · Share on FacebookShare on Twitter
  • AdrienL
    AdrienL Dataiker, Alpha Tester Posts: 196 Dataiker
    The same error as in the backend logs (which was the one I was addressing, about a *local* EDA kernel) or the same error as in you original post's main text (about a *docker* EDA kernel)?
    If it's the latter, try unsetting docker configuration and using a local kernel to check whether this works.
    · Share on FacebookShare on Twitter
  • Jailbreaker
    Jailbreaker Registered Posts: 9 ✭✭✭✭

    Even with the "local" Kernel, I get the same error

    Unexpected EDA kernel failure

    com.dataiku.dip.exceptions.CodedException: EDA local kernel failed to start, caused by: CodedException: EDA local kernel failed to start, caused by: SecretKernelTimeoutException: Subprocess failed to connect, it probably crashed at startup. Check the logs., caused by: SocketException: Socket closed
    · Share on FacebookShare on Twitter
  • AdrienL
    AdrienL Dataiker, Alpha Tester Posts: 196 Dataiker
    Yes, this is the same overall error, the details are in the backend logs you attached the first time, please attache those again.
    · Share on FacebookShare on Twitter
  • Jailbreaker
    Jailbreaker Registered Posts: 9 ✭✭✭✭

    It's the same error I think, as you can see in the attached log file

    · Share on FacebookShare on Twitter
  • AdrienL
    AdrienL Dataiker, Alpha Tester Posts: 196 Dataiker
    Strange. Can you try, in a local notebook in the built-in environment, to `import sortedcontainer`?
    · Share on FacebookShare on Twitter
  • Jailbreaker
    Jailbreaker Registered Posts: 9 ✭✭✭✭

    Seems that I add it in the python_env, not in the built in Environment!

    Because is the Jupyter Notebook, by default, it doesn't import it. But when I change the kernel to my python_environment, all is OK.

    What I want is to add this package 'sortedcontainers' to my built in environement!

    So how can I add it to the built in Environment so?

    · Share on FacebookShare on Twitter
  • AdrienL
    AdrienL Dataiker, Alpha Tester Posts: 196 Dataiker
    I am not sure I understand what you mean. I think the easiest would be to share an instance diagnostic with support@dataiku.com (from Administration > Maintenance > Diagnostic tool). If the resulting file is too large for mail (> 15 MB), you can use https://dl.dataiku.com to send it. If so, please don't forget to send the link that is generated when you upload the file.
    · Share on FacebookShare on Twitter
  • Jailbreaker
    Jailbreaker Registered Posts: 9 ✭✭✭✭

    OK

    · Share on FacebookShare on Twitter
  • AdrienL
    AdrienL Dataiker, Alpha Tester Posts: 196 Dataiker
    You should send that to support@dataiku.com (or on support.dataiku.com), as these files can contain things you may not want public, and support is better armed than community to answer on the specific configuration that you have and that is not common to other users.
    · Share on FacebookShare on Twitter
  • Masoud
    Masoud Registered Posts: 1 ✭

    i recive this error:

    D_server_van DSS.docx] - Calling Structured extractor with settings: maxSectionDepth=6, imageHandlingMode=IGNORE[15:56:38] [ERROR] [dku.recipes.nlp.embed_documents_applier] - [/Handleiding_Azure Devops-Git repository -code studio.docx] - Got structured extraction result: failed - Kubernetes kernel failed to start, caused by: SecretKernelTimeoutException: Subprocess did not connect in 60000ms, it probably crashed at startup. Check the logs., caused by: SocketTimeoutException: Accept timed out (processing time: 130500ms)[15:56:38] [ERROR] [dku.recipes.nlp.embed_documents_applier] - [/Handleiding_proceces-mining-Maak_een_instance.docx] - Got structured extraction result: failed - Kubernetes kernel failed to start, caused by: SecretKernelTimeoutException: Subprocess did not connect in 60000ms, it probably crashed at startup. Check the logs., caused by: SocketTimeoutException: Accept timed out (processing time: 130495ms)[15:56:38] [ERROR] [dku.recipes.nlp.embed_documents_applier] - [/Handleiding_code studio-CodeEnvs-jobs.docx] - Got structured extraction result: failed - Kubernetes kernel failed to start, caused by: SecretKernelTimeoutException: Subprocess did not connect in 60000ms, it probably crashed at startup. Check the logs., caused by: SocketTimeoutException: Accept timed out (processing time: 130497ms)[15:56:38] [ERROR] [dku.recipes.nlp.embed_documents_applier] - [/Handleiding_proceces-mining-Maak_een_instance.docx] - Document failed to be processed and was skippedjava.util.concurrent.CompletionException: java.lang.RuntimeException: java.lang.Exception: Kubernetes kernel failed to start, caused by: SecretKernelTimeoutException: Subprocess did not connect in 60000ms, it probably crashed at startup. Check the logs., caused by: SocketTimeoutException: Accept timed out

    · Share on FacebookShare on Twitter
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…