[Resolved] Coder Love: Support for JupyterLab

importthepandas
importthepandas Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 115 Neuron
in Product Ideas

[Resolved] Thanks to the Code Studios feature in Dataiku 11, you can now use JupyterLab to edit (and debug!) Python recipes and libraries.

Checkout this video that shows how it looks like:
https://content.dataiku.com/dataiku-11/code-studios

Code Studios is documented here:
https://doc.dataiku.com/dss/latest/code-studios/index.html

And to learn how to run JupyterLab in DSS using a Code Studio template, you can read this paragraph:
https://doc.dataiku.com/dss/latest/code-studios/code-studio-templates.html#sample-deploying-jupyterl...

Post:
JupyterLab will eventually replace Jupyter notebook. Lab offers a much more IDE like experience with better collaboration for coders, including debuggers and other extensions that help coders tremendously. Notebook interfaces are notorious for producing dirty code. While JupyterLab is not perfect, it helps to organize code and supporting files and folders more effectively than a standalone notebook. It would enhance the quality of code in python recipes on the flow due to better documentation features and markdown + console support in a single view. Consoles would allow for more effective testing and keep books clean. My team spends a ton of time surfing through old code and test code in notebooks. Thus we spend a significant amount of time "productionizing" code in Dataiku before projects can be merged to main and deployed to auto.

Default features like expanding and collapsing, dragging and dropping cells help organize and enhance collaboration. Side by side views and window organization help coders to be more effective on the Dataiku platform.

One key feature that's highly enjoyable for collaboration is kernel sharing. My team is global and all virtual now because of COVID. With standard Jupyter notebooks, our data science leads need to re-run time intensive/costly cells to catch up to a point where an analyst or another colleague needs help.

A user could push various notebooks in a lab session to the flow sequentially. One potential aspect to account for is the inherent filesystem.

Tagged:
Share on FacebookShare on Twitter
15
15 votes

Released · Last Updated

Comments

  • xavsun
    xavsun Registered Posts: 4 ✭✭✭✭

    This will be a killer feature!!! Dataiku - please!!

    · Share on FacebookShare on Twitter
  • Ignacio_Toledo
    Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 415 Neuron

    I support the idea to integrate jupyterlabs into DSS, specially since apparently it will replace the Jupyter notebooks. However I do not share the other reasons.

    Just my opinion of course, but notebooks are great places to experiment or to share results, or add context and documentation to a data science problem and its solutions, but they are not a place were libraries should be developed.

    I'm kind of one sided in this view because this is how we have separated the realms of the software development team from the data analytics/science team in our organization, and we have been able to move from a useless competition to a fruitful collaboration. Today, data analyst and scientists are learning to use DSS to make the analytical workflow happen, and software developers are creating production level code that is shared with the analyst through the "Libraries" section of a DSS Project. Jupyter notebooks are in general used to load these libraries and apply them to the datasets in the flow, or to share reports.

    Then again, this is just my opinion, or at best one way to organize the analytical work. So please do not take this comment as the right answer or anything like that!

    · Share on FacebookShare on Twitter
  • Ashley
    Ashley Dataiker, Alpha Tester, Dataiku DSS Core Designer, Registered, Product Ideas Manager Posts: 162 Dataiker

    Hi @importthepandas
    ,

    Thanks for submitting this idea and for sharing the context around why it would be useful to your team. You'll be pleased to hear this idea is in our backlog--integrating w/ JupyterLab is a request we've received from customers--and we are determining the next steps for development. We can't provide a timeline at this point, but be sure to check back for updates.

    Best,

    Ashley

    · Share on FacebookShare on Twitter
  • importthepandas
    importthepandas Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 115 Neuron

    thanks Ashely! looking forward to more support in this space

    · Share on FacebookShare on Twitter
  • CoreyS
    CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭

    While we continue to explore this idea, we just wanted to make you aware that as of version 9.0.2 we now have eased access to install nbextensions with Using Jupyter Widgets

    · Share on FacebookShare on Twitter
  • Ignacio_Toledo
    Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 415 Neuron

    @CoreyS
    I saw this mentioned as a new improvement, however I couldn't find documentation on how to actually install the nbextensions. Do wyou have any pointers or tips?

    Thanks!

    · Share on FacebookShare on Twitter
  • Ignacio_Toledo
    Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 415 Neuron
    edited July 17

    @CoreyS
    , I did some looking around in DSS data_dir of my installation, and I was able to found this new option for dssadmin:

    bin/dssadmin jupyter-nbextensions

    When looking for the help of this new option I've got this instruction to use it

    jupyter-nbextensions ACTION (list, enable, disable, available) [extension/main]

    I think what is missing currently in the official documentation is this new option, do you think it could be added?

    Cheers!

    · Share on FacebookShare on Twitter
  • MarcSkovMadsen
    MarcSkovMadsen Registered Posts: 3 ✭✭✭

    Definitely this would be a nice edition.

    And please make sure the Voila and Panel Jupyter Preview is enabled.

    https://blog.holoviz.org/panel_0.12.0.html#JupyterLab-previews

    · Share on FacebookShare on Twitter
  • fsergot
    fsergot Dataiker, Registered, Product Ideas Manager Posts: 118 Dataiker

    Hello everyone,

    I am pleased to announced that we are working on a solution to allow DSS users to code in JupyterLab.

    The first version of this new capabilities is currently planned for Q2 this year so stay tuned.

    · Share on FacebookShare on Twitter
  • importthepandas
    importthepandas Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 115 Neuron

    @fsergot
    you guys are amazing, thank you

    · Share on FacebookShare on Twitter
  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,088 Neuron

    Well it looks like JupyterLab didn't make the v11 cut. By v11 I mean the Dataiku version that has been announced but not yet released

    https://discover.dataiku.com/dataiku-11/

    https://www.dataiku.com/product/product-updates/

    Personally I think Code Studios is a much better approach than integrating JupyterLab. But I will be interested to know if JupyterLab is still coming and what features are still missing now that v11 "will" have Code Studios.

    · Share on FacebookShare on Twitter
  • Ignacio_Toledo
    Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 415 Neuron

    Hi @Turribeach

    Well, I think there is hope it will be there, as it shown on the new features videos. See the screen shot here:

    Jupterlab template in "Code Studios"

    Cheers

    · Share on FacebookShare on Twitter
  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,088 Neuron

    Ah nice, we shall have a look when v11 comes out (in the real world not the PR world)

    · Share on FacebookShare on Twitter
  • fsergot
    fsergot Dataiker, Registered, Product Ideas Manager Posts: 118 Dataiker

    Hello,

    Just a quick update following the latest comments. JupyterLab is indeed part of a new feature called Code Studio. In fact Code Studio will allow to run your own web IDE managed by DSS, it can be JupyterLab but also RStudio or VSCode (to which we will provide builtin templates so the setup should be very simple).

    v11 is not far away at all so be ready

    · Share on FacebookShare on Twitter
  • apichery
    apichery Dataiker, Alpha Tester, Registered, Product Ideas Manager, Moderator Posts: 64 Dataiker

    You asked for it... we did it!

    Thanks to the newly introduced Code Studios feature in Dataiku 11, you can now use JupyterLab to edit (and debug!) Python recipes and libraries.

    Checkout this video that shows how it looks like:
    https://content.dataiku.com/dataiku-11/code-studios

    Code Studios is documented here:
    https://doc.dataiku.com/dss/latest/code-studios/index.html

    And to learn how to run JupyterLab in DSS using a Code Studio template, you can read this paragraph:
    https://doc.dataiku.com/dss/latest/code-studios/code-studio-templates.html#sample-deploying-jupyterlab

    · Share on FacebookShare on Twitter
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…