Replace the code editor
Whether it's Python, R, SQL, Scala, Pig, etc., many Dataiku users spend quite a bit of time writing code. Generally, users writing sophisticated code will want to do so either in a notebook or in Visual Studio Code using the Dataiku plugin rather than in the web browser, since the editor is very limited. Features like code completion and debugging are simply not available, and the current development process typically involves jumping back and forth between the editor page and logs.
It'd be amazing if a complete code editor such as Code Server (Visual Studio Code for web browsers) were integrated into Dataiku. Code completion and function documentation would reduce how often users need to reference the full API docs. Project integrations such as those already included in the Dataiku VS Code plugin could make referencing datasets very quick (instead of the current process of switching to the Input/Output tab). Instead of creating single-file scripts, code recipes could leverage multiple files to keep code organized. And users could write commit messages with their changes, enabling better collaboration.This editor could also leverage existing VS Code language engine plugins for syntax validation and highlighting, which would make SQL editing specific to the database engine. Instead of having multiple windows for logs and code, code output could be written directly to a pane in the editor, consistent with a desktop editor experience. Sample data output could also be written to a pane in the editor, allowing users to see results in real time (just like the Prepare recipe already does, but ideally not a full screen UX). Tying in a debugger (where language support is available) could enable insights into exactly what's happening, allowing users to step through their code and understand exactly how data is being transformed.
This could also transform the way notebooks are created, since VS Code plugins for notebooks tend to provide a much more sophisticated environment than the vanilla web UIs, bringing the best editor features (like intellisense) to the notebook editing experience.
Finally, this would allow for some nice customization features, such as theming (even if exposed only through json files or the built-in VS Code settings manager), allowing users who spend much of their time writing code the most comfortable experience.
An editor like this would integrate nicely with @Marlan
's proposed feature as well.
Comments
-
Thanks for your idea, @natejgardner !
Your idea meets the criteria for submission, we'll reach out should we require more information.
-
Hello,
I am happy to say that we have acknowledged the need for more freedom of choice regarding coding in DSS and are actually working on a solution.
The idea would be to allow a DSS administrator to configure custom coding environments and make them available to end users. Through this environments, end users would be able to leverage any IDE they want, or use a customized stack for their WebApps. It would come at a price, which is to define those environments but we are targeting to deliver pre-packaged environments for the most requested use cases (like JupyterLab, RStudio or Streamlit webApp - this list being not exhaustive not being a commitment
). I will let you when this is shipped.
-
Linking this to
-
You asked for it... we did it!
Thanks to the newly introduced Code Studios feature in Dataiku 11, you can now use Visual Studio Code to edit (and debug!) Python recipes and libraries. It's not as deeply integrated into DSS as suggested, but it really allows coders to unleash the power of Visual Studio Code to code into DSS.Checkout this video that shows how it looks like:
https://content.dataiku.com/dataiku-11/code-studios
Code Studios is documented here:
https://doc.dataiku.com/dss/latest/code-studios/index.html
And if you want to try it, just follow this tutorial:
https://knowledge.dataiku.com/latest/courses/advanced-code/work-environment/code-studios-debug.html -
We just released a new tutorial on how to build and use your first Code Studio, do not hesitate to follow it: Hands-On Tutorial: My First Code Studio
Another one is coming on how to use Code Studio to deploy a custom webapp using Streamlit
