Executing notebooks inside a container with UIF activated
Hello,
I tried running a jupyter notebook's kernel inside a container while UIF was enabled. And I'm getting this error in the kernel:
Traceback (most recent call last):
File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/web.py", line 1592, in _execute
result = yield result
File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run
value = future.result()
File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1141, in run
yielded = self.gen.throw(*exc_info)
File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/services/sessions/handlers.py", line 81, in post
user=self.get_current_user(),
File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run
value = future.result()
File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1141, in run
yielded = self.gen.throw(*exc_info)
File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/dataiku/sessionmanager.py", line 87, in create_session
DKU_EXTRA_ENV= dku_extra_env)
File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run
value = future.result()
File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1141, in run
yielded = self.gen.throw(*exc_info)
File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/services/sessions/sessionmanager.py", line 96, in start_kernel_for_session
self.kernel_manager.start_kernel(path=kernel_path, kernel_name=kernel_name, **kwargs)
File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run
value = future.result()
File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 326, in wrapper
yielded = next(result)
File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/services/kernels/kernelmanager.py", line 148, in start_kernel
super(MappingKernelManager, self).start_kernel(**kwargs)
File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/jupyter_client/multikernelmanager.py", line 110, in start_kernel
km.start_kernel(**kwargs)
File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/jupyter_client/manager.py", line 259, in start_kernel
**kw)
File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/dataiku/kernelmanager.py", line 173, in _launch_kernel
filename = self.build_description(kernel_cmd, self.impersonate, kernel_cwd, env, kernel_context)
File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/dataiku/kernelmanager.py", line 106, in build_description
"dss_user" : kernel_context.get('dssUser', os.environ['USER']),
File "/usr/lib64/python3.6/os.py", line 669, in __getitem__
raise KeyError(key) from None
KeyError: 'USER'
When I don't activate the UIF I have no problem. So I don't understand why this happens when I enable, despite not trying to run any code locally.
Answers
-
Hi,
Could you share what code you are trying to run in your notebook? Also, if you run the same in a python recipe, do you get similar errors? If so, please open a support ticket with the corresponding job diagnosis.
Thanks,
Andrew
-
I'm running the default generated code whenever you create a new notebook. This issue does not occur while running a python recipe. Those work as expected even when UIF is enabled.
-
The error you are receiving implies that the USER is not properly set on the instance or at least it's not being successfully retrieved from the OS environment variables (os.environ['USER']). If possible, please try reviewing and printing out the OS environment variables to make sure they are configured appropriately.
You can potentially try working around the issue by adding the following to the <DATA_DIR>/bin/env-site.sh file:
export USER=<insert_user>
-
I tried setting that variable manually. But I don't understand why is it needed? Why is the UIF relevant when executing the kernel in a container??
And adding the variable manually moves me to this next error:
Traceback (most recent call last): File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/web.py", line 1592, in _execute result = yield result File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run value = future.result() File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1141, in run yielded = self.gen.throw(*exc_info) File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/services/sessions/handlers.py", line 81, in post user=self.get_current_user(), File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run value = future.result() File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1141, in run yielded = self.gen.throw(*exc_info) File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/dataiku/sessionmanager.py", line 87, in create_session DKU_EXTRA_ENV= dku_extra_env) File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run value = future.result() File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1141, in run yielded = self.gen.throw(*exc_info) File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/services/sessions/sessionmanager.py", line 96, in start_kernel_for_session self.kernel_manager.start_kernel(path=kernel_path, kernel_name=kernel_name, **kwargs) File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 1133, in run value = future.result() File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/tornado/gen.py", line 326, in wrapper yielded = next(result) File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/services/kernels/kernelmanager.py", line 148, in start_kernel super(MappingKernelManager, self).start_kernel(**kwargs) File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/jupyter_client/multikernelmanager.py", line 110, in start_kernel km.start_kernel(**kwargs) File "/home/dataiku/dataiku-dss-10.0.5/python36.packages/jupyter_client/manager.py", line 259, in start_kernel **kw) File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/dataiku/kernelmanager.py", line 201, in _launch_kernel check_alive() File "/home/dataiku/dataiku-dss-10.0.5/dku-jupyter/packages/notebook/dataiku/kernelmanager.py", line 187, in check_alive raise Exception("MUS Wrapper process died !") Exception: MUS Wrapper process died !I don't want UIF to be relevant at all when I'm using containerized execution. This works correctly for any recipe. But it is broken in notebooks. Is this intentional?
