Executing notebooks inside a container with UIF activated

DrissiReda
DrissiReda Registered Posts: 57 ✭✭✭✭✭
edited July 16 in Using Dataiku

Hello,

I tried running a jupyter notebook's kernel inside a container while UIF was enabled. And I'm getting this error in the kernel:

Traceback (most recent call last):
  File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/web.py", line 1592, in _execute
    result = yield result
  File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1141, in run
    yielded = self.gen.throw(*exc_info)
  File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/services/sessions/handlers.py", line 81, in post
    user=self.get_current_user(),
  File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1141, in run
    yielded = self.gen.throw(*exc_info)
  File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/dataiku/sessionmanager.py", line 87, in create_session
    DKU_EXTRA_ENV= dku_extra_env)
  File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1141, in run
    yielded = self.gen.throw(*exc_info)
  File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/services/sessions/sessionmanager.py", line 96, in start_kernel_for_session
    self.kernel_manager.start_kernel(path=kernel_path, kernel_name=kernel_name, **kwargs)
  File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 326, in wrapper
    yielded = next(result)
  File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/services/kernels/kernelmanager.py", line 148, in start_kernel
    super(MappingKernelManager, self).start_kernel(**kwargs)
  File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/jupyter_client/multikernelmanager.py", line 110, in start_kernel
    km.start_kernel(**kwargs)
  File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/jupyter_client/manager.py", line 259, in start_kernel
    **kw)
  File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/dataiku/kernelmanager.py", line 173, in _launch_kernel
    filename = self.build_description(kernel_cmd, self.impersonate, kernel_cwd, env, kernel_context)
  File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/dataiku/kernelmanager.py", line 106, in build_description
    "dss_user" : kernel_context.get('dssUser', os.environ['USER']),
  File "/usr/lib64/python3.6/os.py", line 669, in __getitem__
    raise KeyError(key) from None
KeyError: 'USER'

When I don't activate the UIF I have no problem. So I don't understand why this happens when I enable, despite not trying to run any code locally.

Answers

  • ATsao
    ATsao Dataiker Alumni, Registered Posts: 139 ✭✭✭✭✭✭✭✭

    Hi,

    Could you share what code you are trying to run in your notebook? Also, if you run the same in a python recipe, do you get similar errors? If so, please open a support ticket with the corresponding job diagnosis.

    Thanks,

    Andrew

  • DrissiReda
    DrissiReda Registered Posts: 57 ✭✭✭✭✭

    I'm running the default generated code whenever you create a new notebook. This issue does not occur while running a python recipe. Those work as expected even when UIF is enabled.

  • ATsao
    ATsao Dataiker Alumni, Registered Posts: 139 ✭✭✭✭✭✭✭✭
    edited July 17

    The error you are receiving implies that the USER is not properly set on the instance or at least it's not being successfully retrieved from the OS environment variables (os.environ['USER']). If possible, please try reviewing and printing out the OS environment variables to make sure they are configured appropriately.

    You can potentially try working around the issue by adding the following to the <DATA_DIR>/bin/env-site.sh file:

    export USER=<insert_user>

  • DrissiReda
    DrissiReda Registered Posts: 57 ✭✭✭✭✭
    edited July 17

    I tried setting that variable manually. But I don't understand why is it needed? Why is the UIF relevant when executing the kernel in a container??

    And adding the variable manually moves me to this next error:

    Traceback (most recent call last):
      File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/web.py", line 1592, in _execute
        result = yield result
      File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run
        value = future.result()
      File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1141, in run
        yielded = self.gen.throw(*exc_info)
      File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/services/sessions/handlers.py", line 81, in post
        user=self.get_current_user(),
      File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run
        value = future.result()
      File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1141, in run
        yielded = self.gen.throw(*exc_info)
      File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/dataiku/sessionmanager.py", line 87, in create_session
        DKU_EXTRA_ENV= dku_extra_env)
      File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run
        value = future.result()
      File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1141, in run
        yielded = self.gen.throw(*exc_info)
      File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/services/sessions/sessionmanager.py", line 96, in start_kernel_for_session
        self.kernel_manager.start_kernel(path=kernel_path, kernel_name=kernel_name, **kwargs)
      File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run
        value = future.result()
      File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 326, in wrapper
        yielded = next(result)
      File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/services/kernels/kernelmanager.py", line 148, in start_kernel
        super(MappingKernelManager, self).start_kernel(**kwargs)
      File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/jupyter_client/multikernelmanager.py", line 110, in start_kernel
        km.start_kernel(**kwargs)
      File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/jupyter_client/manager.py", line 259, in start_kernel
        **kw)
      File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/dataiku/kernelmanager.py", line 201, in _launch_kernel
        check_alive()
      File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/dataiku/kernelmanager.py", line 187, in check_alive
        raise Exception("MUS Wrapper process died !")
    Exception: MUS Wrapper process died !

    I don't want UIF to be relevant at all when I'm using containerized execution. This works correctly for any recipe. But it is broken in notebooks. Is this intentional?

Setup Info
    Tags
      Help me…