Spotlight on New Gen AI Capabilities
Dataiku 12.4: The Gift That Keeps On Giving, Part Deux
Hi everyone, me again!
Hopefully, you saw last week's update that Dataiku 12.4 is officially available and chock full of goodies to help you with multi-platform MLOps, statistics, visualization, modeling, and building Flows. If not, check out the article now.
But wait, there’s more! With the holiday season in full swing, I know I can’t be the only one wishing for a team of magical elves to help get things done. You, too? Good news: In Dataiku 12.4, we introduced three LLM-powered AI Assistants in private preview that not only will boost your efficiency, but may also improve the overall quality of your data products and teach you a few things along the way.
AI Code Assistants
Let’s be honest; when developing code for data science projects, many of us spend a lot of time researching how to do specific tasks, debugging or refactoring code, or creating unit tests. Generative AI-based code assistants are now available in Dataiku for Jupyter Notebooks and VSCode Code Studios to enhance and accelerate the coding experience. Ask the assistant to write new code from natural language instructions or from a highlighted comment, explain existing code, refactor or find problems in the code, create unit tests, and more. Or, submit a question or magic command through the AI Code Assistant and receive answers that are enriched with contextual knowledge of your code and previously asked questions.
//play.vidyard.com/zVqLPMinWHWhPpK7oQMPwN.html?
How does it work under the hood? Dataiku’s AI Code Assistants leverage LLM Mesh connections to many LLM providers, such as OpenAI (which powers GitHub Copilot), Cohere, or Hugging Face. Your Dataiku administrator will first enable the feature and select the default LLM to use for the assistant, then you can access the feature from the embedded Jupyter Notebooks or VS Code Code Studios. Coders rejoice, and say hello to your new favorite shortcut!
AI Explain
I’m not sure what’s more tedious — painstakingly investigating individual elements in a pipeline to grasp the purpose and rationale behind a Flow, or being the one who actually has to manually document and maintain project and Flow descriptions! Although documentation is critical to preserving business and technical context for future team members or project stakeholders, this time-consuming task often goes neglected or unmaintained.
Why not instead use AI Explain, Dataiku’s Generative AI-powered assistant, to automatically generate descriptions that explain Dataiku Flows or Flow zones?
//play.vidyard.com/6jdiMa6PHLCcfnMxbkuPhc.html?
You can configure the language, intended purpose, and length of the explanations, and freely edit the suggested text before saving as a project artifact. Use AI Explain to get a quick primer of what’s going on in a Dataiku project you’ve just gotten access to, or take advantage of its eloquence and knowledge of the project to bootstrap descriptions that will help others understand what’s going on. Don’t worry; Individual rows of data are not shared to the LLM, only project metadata and dataset schemas and descriptions.
AI Prepare
We all know that data prep comprises a large proportion of time spent on any data project. Knowing what you want to do to your data is one thing, but knowing HOW to efficiently do it is quite another! With over a hundred processors available in the prepare recipe, it’s inefficient to cleanse, prepare, or transform data by trial and error…and perhaps a daunting task if you’re a less technical person unsure of which processors to apply.
With Dataiku’s AI Prepare assistant, you can work smarter, not harder. Simply describe the transformation you want to apply in natural language and the AI assistant automatically generates the necessary data preparation steps. The ability to modify both your prompt and the resulting steps means you can prepare data faster than ever, yet still stay in complete control.
I’m confident every single user of Dataiku can benefit from these productivity boosters, and enjoy a fun and more satisfying working experience as an added bonus. Contact your Dataiku rep or Customer Success Manager for details on how to enable these features in private preview. We hope you enjoy the new AI Assistants, and let us know what you think in the comments!