Improved SQL Experience

M1-Nick Registered Posts: 11 ✭✭

It would be really nice if the SQL experience felt more like it was a natural part of the workflow.

Right now when you start a blank project, the flow experiences pushes you to first add a dataset. If that dataset is in a databases, you are asked to select your table to create your dataset. If you select writing with a custom SQL query, you are given a pretty distinct warning not to use this and a small box to enter some sql into. This most likely means you aren't starting here with SQL.

Another alternative for this is to just select a random table as the dataset so that you can then add a SQL recipe to your flow and work from there.

If you go to the notebook and start with a SQL notebook, you get a pretty nice SQL editor, with features like explain plans etc.

These experiences feel super disjointed and very confusing.

What would be nice is if these these experience were combined and we were able to start with the SQL notebook, write the SQL and then move from that view to back to the flow or into a Python/ R notebook.

It would also be great if we could take the SQL query, version control that into Github as a saved query for others to use.

15 votes

Released · Last Updated


  • CoreyS
    CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭

    Hi @M1-Nick
    thank you for your idea. I was wondering if you felt this was similar to @Marlan
    's idea of ? If so I'd be happy to merge these two ideas together.

  • M1-Nick
    M1-Nick Registered Posts: 11 ✭✭

    I think they are similar yeah, but not the same.

    talks about doing jinja tempting and changing how the code is written.

    I think my suggestion is more around how the various SQL options fit into the overall worksteam of Dataiku. Basically shifting things around more and making the way the options are presented to the user more seamless.

    I think combining them would be something that could be done if you want to completely redefine the SQL experience, otherwise they are could probably be tackled as separate requests.

  • Marlan
    Marlan Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Dataiku Frontrunner Awards 2021 Participant, Neuron 2023 Posts: 317 Neuron

    Hi @CoreyS
    , to weigh in here too... I agree with @M1-Nick
    that the ideas are different and should remain separate.

    That said, I like these ideas. My typical flow in a new project is to start with a code recipe (SQL Script) and yes it would be great to be able to work between SQL recipes (probably most easily SQL Script recipes) and SQL Notebooks just like we can with Python recipes.


  • Katie
    Katie Dataiker, Registered, Product Ideas Manager Posts: 105 Dataiker

    Hey @M1-Nick
    ! Fancy meeting you here

    Thanks for the feedback, we've logged it internally and will let you know if we have updates on any of these topics.

  • M1-Nick
    M1-Nick Registered Posts: 11 ✭✭

    - sounds good! You know where to find me if you have questions!

  • AshleyW
    AshleyW Dataiker, Alpha Tester, Dataiku DSS Core Designer, Registered, Product Ideas Manager Posts: 161 Dataiker


    Updating this thread to let you know that with the release of 12.5 we have a more fluid SQL recipe <--> Notebook experience: SQL recipes can now be edited in Notebooks, just like Python recipes can!

    When you open a SQL recipe, you'll see a button, 'Edit in Notebook', that lets you do just that.



Setup Info
      Help me…