[Resolved] Coder Love: Support for JupyterLab

importthepandas

[Resolved] Thanks to the Code Studios feature in Dataiku 11, you can now use JupyterLab to edit (and debug!) Python recipes and libraries.

Checkout this video that shows how it looks like:
https://content.dataiku.com/dataiku-11/code-studios

Code Studios is documented here:
https://doc.dataiku.com/dss/latest/code-studios/index.html

And to learn how to run JupyterLab in DSS using a Code Studio template, you can read this paragraph:
https://doc.dataiku.com/dss/latest/code-studios/code-studio-templates.html#sample-deploying-jupyterl...

Post:
JupyterLab will eventually replace Jupyter notebook. Lab offers a much more IDE like experience with better collaboration for coders, including debuggers and other extensions that help coders tremendously. Notebook interfaces are notorious for producing dirty code. While JupyterLab is not perfect, it helps to organize code and supporting files and folders more effectively than a standalone notebook. It would enhance the quality of code in python recipes on the flow due to better documentation features and markdown + console support in a single view. Consoles would allow for more effective testing and keep books clean. My team spends a ton of time surfing through old code and test code in notebooks. Thus we spend a significant amount of time "productionizing" code in Dataiku before projects can be merged to main and deployed to auto.

Default features like expanding and collapsing, dragging and dropping cells help organize and enhance collaboration. Side by side views and window organization help coders to be more effective on the Dataiku platform.

One key feature that's highly enjoyable for collaboration is kernel sharing. My team is global and all virtual now because of COVID. With standard Jupyter notebooks, our data science leads need to re-run time intensive/costly cells to catch up to a point where an analyst or another colleague needs help.

A user could push various notebooks in a lab session to the flow sequentially. One potential aspect to account for is the inherent filesystem.

xavsun

This will be a killer feature!!! Dataiku - please!!

Ignacio_Toledo

I support the idea to integrate jupyterlabs into DSS, specially since apparently it will replace the Jupyter notebooks. However I do not share the other reasons.

Just my opinion of course, but notebooks are great places to experiment or to share results, or add context and documentation to a data science problem and its solutions, but they are not a place were libraries should be developed.

I'm kind of one sided in this view because this is how we have separated the realms of the software development team from the data analytics/science team in our organization, and we have been able to move from a useless competition to a fruitful collaboration. Today, data analyst and scientists are learning to use DSS to make the analytical workflow happen, and software developers are creating production level code that is shared with the analyst through the "Libraries" section of a DSS Project. Jupyter notebooks are in general used to load these libraries and apply them to the datasets in the flow, or to share reports.

Then again, this is just my opinion, or at best one way to organize the analytical work. So please do not take this comment as the right answer or anything like that!

AshleyW

Hi @importthepandas
,

Thanks for submitting this idea and for sharing the context around why it would be useful to your team. You'll be pleased to hear this idea is in our backlog--integrating w/ JupyterLab is a request we've received from customers--and we are determining the next steps for development. We can't provide a timeline at this point, but be sure to check back for updates.

Best,

Ashley

importthepandas

thanks Ashely! looking forward to more support in this space

CoreyS

While we continue to explore this idea, we just wanted to make you aware that as of version 9.0.2 we now have eased access to install nbextensions with Using Jupyter Widgets

Ignacio_Toledo

@CoreyS
I saw this mentioned as a new improvement, however I couldn't find documentation on how to actually install the nbextensions. Do wyou have any pointers or tips?

Thanks!

Ignacio_Toledo

@CoreyS
, I did some looking around in DSS data_dir of my installation, and I was able to found this new option for dssadmin:

bin/dssadmin jupyter-nbextensions

When looking for the help of this new option I've got this instruction to use it

jupyter-nbextensions ACTION (list, enable, disable, available) [extension/main]

I think what is missing currently in the official documentation is this new option, do you think it could be added?

Cheers!

MarcSkovMadsen

Definitely this would be a nice edition.

And please make sure the Voila and Panel Jupyter Preview is enabled.

https://blog.holoviz.org/panel_0.12.0.html#JupyterLab-previews

fsergot

Hello everyone,

I am pleased to announced that we are working on a solution to allow DSS users to code in JupyterLab.

The first version of this new capabilities is currently planned for Q2 this year so stay tuned.

importthepandas

@fsergot
you guys are amazing, thank you

Turribeach

Well it looks like JupyterLab didn't make the v11 cut. By v11 I mean the Dataiku version that has been announced but not yet released

https://discover.dataiku.com/dataiku-11/

https://www.dataiku.com/product/product-updates/

Personally I think Code Studios is a much better approach than integrating JupyterLab. But I will be interested to know if JupyterLab is still coming and what features are still missing now that v11 "will" have Code Studios.

Ignacio_Toledo

Hi @Turribeach

Well, I think there is hope it will be there, as it shown on the new features videos. See the screen shot here:

Cheers

Turribeach

Ah nice, we shall have a look when v11 comes out (in the real world not the PR world)

fsergot

Hello,

Just a quick update following the latest comments. JupyterLab is indeed part of a new feature called Code Studio. In fact Code Studio will allow to run your own web IDE managed by DSS, it can be JupyterLab but also RStudio or VSCode (to which we will provide builtin templates so the setup should be very simple).

v11 is not far away at all so be ready

apichery

You asked for it... we did it!

Thanks to the newly introduced Code Studios feature in Dataiku 11, you can now use JupyterLab to edit (and debug!) Python recipes and libraries.

Checkout this video that shows how it looks like:
https://content.dataiku.com/dataiku-11/code-studios

Code Studios is documented here:
https://doc.dataiku.com/dss/latest/code-studios/index.html

And to learn how to run JupyterLab in DSS using a Code Studio template, you can read this paragraph:
https://doc.dataiku.com/dss/latest/code-studios/code-studio-templates.html#sample-deploying-jupyterlab