On opening/loading, python notebook gives Out of Memory error

Anu

Hi,

When I try to open a python notebook, screen freezes after couple of seconds and finally throws error saying "Out of Memory". Any suggestions to resolve this issue is appreciated. Thanks.

Regards,

AJ

JordanB

Hi @Anu
,

This usually means that the kernel is using more memory than available either for the container configuration or allowed via cgroup if running locally. You can get a better idea of the reason for the failure if you can execute the same code in a python recipe instead of a notebook and check the output log.

I recommend also checking your running background tasks by going to Administration > Monitoring > Running background tasks and unload any tasks that may have been left running. Note, you must be an admin to do this.

If you continue to receive this error, please email or live chat support, and attach the instance diagnostic (Administration > Maintenance > Instance diagnosis).

Thanks!

Jordan

Jack_M

Hi there. I know this is an old thread, but posting my fix for future visibility.

I had the same issue where a cell output was so large that it couldn't fit into my browser's memory. It was impossible to load the notebook without the tab crashing. This can normally be solved using Kernel → Restart & Clear Output, however sometimes isn't accessible before the page crashes.

From a different python notebook, grab the content of the old one, clear all of the outputs, and then write to a new notebook.

project = dataiku.api_client().get_project('my_project')


nb = project.get_jupyter_notebook('my_notebook')



old_content = nb.get_content()


cells = old_content.get_cells()


n = len(cells)


for i in range(0,n):

    cells[i]['outputs'] = []


#saved = old_content.save()


old_raw = old_content.get_raw()


project.create_jupyter_notebook('my_notebook_2',old_raw)

In newer versions of Dataiku, you might also be able to use the clear_content() function

https://developer.dataiku.com/latest/api-reference/python/other-administration.html#dataikuapi.dss.jupyternotebook.DSSJupyterNotebook

Once you're comfortable that the new notebook has the same cell content as the old, you can delete the old one.