[Resolved] Coder Love: Support for JupyterLab

[Resolved] Thanks to the Code Studios feature in Dataiku 11, you can now use JupyterLab to edit (and debug!) Python recipes and libraries.

Checkout this video that shows how it looks like:
https://content.dataiku.com/dataiku-11/code-studios

Code Studios is documented here:
https://doc.dataiku.com/dss/latest/code-studios/index.html

And to learn how to run JupyterLab in DSS using a Code Studio template, you can read this paragraph:
https://doc.dataiku.com/dss/latest/code-studios/code-studio-templates.html#sample-deploying-jupyterl...

Post:
JupyterLab will eventually replace Jupyter notebook. Lab offers a much more IDE like experience with better collaboration for coders, including debuggers and other extensions that help coders tremendously. Notebook interfaces are notorious for producing dirty code. While JupyterLab is not perfect, it helps to organize code and supporting files and folders more effectively than a standalone notebook. It would enhance the quality of code in python recipes on the flow due to better documentation features and markdown + console support in a single view. Consoles would allow for more effective testing and keep books clean. My team spends a ton of time surfing through old code and test code in notebooks. Thus we spend a significant amount of time "productionizing" code in Dataiku before projects can be merged to main and deployed to auto.

Default features like expanding and collapsing, dragging and dropping cells help organize and enhance collaboration. Side by side views and window organization help coders to be more effective on the Dataiku platform.

One key feature that's highly enjoyable for collaboration is kernel sharing. My team is global and all virtual now because of COVID. With standard Jupyter notebooks, our data science leads need to re-run time intensive/costly cells to catch up to a point where an analyst or another colleague needs help.

A user could push various notebooks in a lab session to the flow sequentially. One potential aspect to account for is the inherent filesystem.

15 Comments
xavsun
Level 2

This will be a killer feature!!! Dataiku - please!!

This will be a killer feature!!! Dataiku - please!!

I support the idea to integrate jupyterlabs into DSS, specially since apparently it will replace the Jupyter notebooks. However I do not share the other reasons.

Just my opinion of course, but notebooks are great places to experiment or to share results, or add context and documentation to a data science problem and its solutions, but they are not a place were libraries should be developed.

I'm kind of one sided in this view because this is how we have separated the realms of the software development team from the data analytics/science team in our organization, and we have been able to move from a useless competition to a fruitful collaboration. Today, data analyst and scientists are learning to use DSS to make the analytical workflow happen, and software developers are creating production level code that is shared with the analyst through the "Libraries" section of a DSS Project. Jupyter notebooks are in general used to load these libraries and apply them to the datasets in the flow, or to share reports.

Then again, this is just my opinion, or at best one way to organize the analytical work. So please do not take this comment as the right answer or anything like that!

I support the idea to integrate jupyterlabs into DSS, specially since apparently it will replace the Jupyter notebooks. However I do not share the other reasons.

Just my opinion of course, but notebooks are great places to experiment or to share results, or add context and documentation to a data science problem and its solutions, but they are not a place were libraries should be developed.

I'm kind of one sided in this view because this is how we have separated the realms of the software development team from the data analytics/science team in our organization, and we have been able to move from a useless competition to a fruitful collaboration. Today, data analyst and scientists are learning to use DSS to make the analytical workflow happen, and software developers are creating production level code that is shared with the analyst through the "Libraries" section of a DSS Project. Jupyter notebooks are in general used to load these libraries and apply them to the datasets in the flow, or to share reports.

Then again, this is just my opinion, or at best one way to organize the analytical work. So please do not take this comment as the right answer or anything like that!

AshleyW
Dataiker

Hi @importthepandas ,

Thanks for submitting this idea and for sharing the context around why it would be useful to your team. You'll be pleased to hear this idea is in our backlog--integrating w/ JupyterLab is a request we've received from customers--and we are determining the next steps for development. We can't provide a timeline at this point, but be sure to check back for updates.

Best,

Ashley

Status changed to: In Backlog

Hi @importthepandas ,

Thanks for submitting this idea and for sharing the context around why it would be useful to your team. You'll be pleased to hear this idea is in our backlog--integrating w/ JupyterLab is a request we've received from customers--and we are determining the next steps for development. We can't provide a timeline at this point, but be sure to check back for updates.

Best,

Ashley

thanks Ashely! looking forward to more support in this space 🙂

thanks Ashely! looking forward to more support in this space 🙂

CoreyS
Dataiker Alumni

While we continue to explore this idea, we just wanted to make you aware that as of version 9.0.2 we now have eased access to install nbextensions with Using Jupyter Widgets

Looking for more resources to help you use Dataiku effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as ‘Accepted Solution’ to help others like you!

While we continue to explore this idea, we just wanted to make you aware that as of version 9.0.2 we now have eased access to install nbextensions with Using Jupyter Widgets

@CoreyS  I saw this mentioned as a new improvement, however I couldn't find documentation on how to actually install the nbextensions. Do wyou have any pointers or tips?

Thanks!

@CoreyS  I saw this mentioned as a new improvement, however I couldn't find documentation on how to actually install the nbextensions. Do wyou have any pointers or tips?

Thanks!

@CoreyS, I did some looking around in DSS data_dir of my installation, and I was able to found this new option for dssadmin:

bin/dssadmin jupyter-nbextensions

When looking for the help of this new option I've got this instruction to use it

jupyter-nbextensions ACTION (list, enable, disable, available) [extension/main]

I think what is missing currently in the official documentation is this new option, do you think it could be added?

Cheers!

 

@CoreyS, I did some looking around in DSS data_dir of my installation, and I was able to found this new option for dssadmin:

bin/dssadmin jupyter-nbextensions

When looking for the help of this new option I've got this instruction to use it

jupyter-nbextensions ACTION (list, enable, disable, available) [extension/main]

I think what is missing currently in the official documentation is this new option, do you think it could be added?

Cheers!

 

MarcSkovMadsen
Level 2

Definitely this would be a nice edition.

And please make sure the Voila and Panel Jupyter Preview is enabled.

https://blog.holoviz.org/panel_0.12.0.html#JupyterLab-previews

 

Definitely this would be a nice edition.

And please make sure the Voila and Panel Jupyter Preview is enabled.

https://blog.holoviz.org/panel_0.12.0.html#JupyterLab-previews

 

fsergot
Dataiker

Hello everyone,

I am pleased to announced that we are working on a solution to allow DSS users to code in JupyterLab.

The first version of this new capabilities is currently planned for Q2 this year so stay tuned.

Status changed to: Developing

Hello everyone,

I am pleased to announced that we are working on a solution to allow DSS users to code in JupyterLab.

The first version of this new capabilities is currently planned for Q2 this year so stay tuned.

@fsergot you guys are amazing, thank you

@fsergot you guys are amazing, thank you