Improved SQL Experience

M1-Nick

It would be really nice if the SQL experience felt more like it was a natural part of the workflow.

Right now when you start a blank project, the flow experiences pushes you to first add a dataset. If that dataset is in a databases, you are asked to select your table to create your dataset. If you select writing with a custom SQL query, you are given a pretty distinct warning not to use this and a small box to enter some sql into. This most likely means you aren't starting here with SQL.

Another alternative for this is to just select a random table as the dataset so that you can then add a SQL recipe to your flow and work from there.

If you go to the notebook and start with a SQL notebook, you get a pretty nice SQL editor, with features like explain plans etc.

These experiences feel super disjointed and very confusing.

What would be nice is if these these experience were combined and we were able to start with the SQL notebook, write the SQL and then move from that view to back to the flow or into a Python/ R notebook.

It would also be great if we could take the SQL query, version control that into Github as a saved query for others to use.

CoreyS

Hi @M1-Nick
thank you for your idea. I was wondering if you felt this was similar to @Marlan
's idea of ? If so I'd be happy to merge these two ideas together.

M1-Nick

@CoreyS
I think they are similar yeah, but not the same.

@Marlan
talks about doing jinja tempting and changing how the code is written.

I think my suggestion is more around how the various SQL options fit into the overall worksteam of Dataiku. Basically shifting things around more and making the way the options are presented to the user more seamless.

I think combining them would be something that could be done if you want to completely redefine the SQL experience, otherwise they are could probably be tackled as separate requests.

Marlan

Hi @CoreyS
, to weigh in here too... I agree with @M1-Nick
that the ideas are different and should remain separate.

That said, I like these ideas. My typical flow in a new project is to start with a code recipe (SQL Script) and yes it would be great to be able to work between SQL recipes (probably most easily SQL Script recipes) and SQL Notebooks just like we can with Python recipes.

Marlan

Katie

Hey @M1-Nick
! Fancy meeting you here

Thanks for the feedback, we've logged it internally and will let you know if we have updates on any of these topics.

M1-Nick

@ktgross15
- sounds good! You know where to find me if you have questions!

Ashley

Hi,

Updating this thread to let you know that with the release of 12.5 we have a more fluid SQL recipe <--> Notebook experience: SQL recipes can now be edited in Notebooks, just like Python recipes can!

When you open a SQL recipe, you'll see a button, 'Edit in Notebook', that lets you do just that.

Cheers,

Ashley