Improved SQL Experience

It would be really nice if the SQL experience felt more like it was a natural part of the workflow.

Right now when you start a blank project, the flow experiences pushes you to first add a dataset. If that dataset is in a databases, you are asked to select your table to create your dataset. If you select writing with a custom SQL query, you are given a pretty distinct warning not to use this and a small box to enter some sql into. This most likely means you aren't starting here with SQL.

Another alternative for this is to just select a random table as the dataset so that you can then add a SQL recipe to your flow and work from there.

If you go to the notebook and start with a SQL notebook, you get a pretty nice SQL editor, with features like explain plans etc.

These experiences feel super disjointed and very confusing.

What would be nice is if these these experience were combined and we were able to start with the SQL notebook, write the SQL and then move from that view to back to the flow or into a Python/ R notebook.

It would also be great if we could take the SQL query, version control that into Github as a saved query for others to use.

6 Comments
CoreyS
Dataiker Alumni

Hi @M1-Nick thank you for your idea. I was wondering if you felt this was similar to @Marlan's idea of Transform the experience of writing SQL? If so I'd be happy to merge these two ideas together. 

Looking for more resources to help you use Dataiku effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as โ€˜Accepted Solutionโ€™ to help others like you!

Hi @M1-Nick thank you for your idea. I was wondering if you felt this was similar to @Marlan's idea of Transform the experience of writing SQL? If so I'd be happy to merge these two ideas together. 

M1-Nick
Level 3

@CoreyS  I think they are similar yeah, but not the same.

@Marlantalks about doing jinja tempting and changing how the code is written.

I think my suggestion is more around how the various SQL options fit into the overall worksteam of Dataiku.  Basically shifting things around more and making the way the options are presented to the user more seamless.

I think combining them would be something that could be done if you want to completely redefine the SQL experience, otherwise they are could probably be tackled as separate requests.

@CoreyS  I think they are similar yeah, but not the same.

@Marlantalks about doing jinja tempting and changing how the code is written.

I think my suggestion is more around how the various SQL options fit into the overall worksteam of Dataiku.  Basically shifting things around more and making the way the options are presented to the user more seamless.

I think combining them would be something that could be done if you want to completely redefine the SQL experience, otherwise they are could probably be tackled as separate requests.

Hi @CoreyS, to weigh in here too... I agree with @M1-Nick that the ideas are different and should remain separate.

That said, I like these ideas. My typical flow in a new project is to start with a code recipe (SQL Script) and yes it would be great to be able to work between SQL recipes (probably most easily SQL Script recipes) and SQL Notebooks just like we can with Python recipes.

Marlan

Hi @CoreyS, to weigh in here too... I agree with @M1-Nick that the ideas are different and should remain separate.

That said, I like these ideas. My typical flow in a new project is to start with a code recipe (SQL Script) and yes it would be great to be able to work between SQL recipes (probably most easily SQL Script recipes) and SQL Notebooks just like we can with Python recipes.

Marlan

ktgross15
Dataiker

Hey @M1-Nick ! Fancy meeting you here ๐Ÿ™‚

Thanks for the feedback, we've logged it internally and will let you know if we have updates on any of these topics.

Status changed to: In the Backlog

Hey @M1-Nick ! Fancy meeting you here ๐Ÿ™‚

Thanks for the feedback, we've logged it internally and will let you know if we have updates on any of these topics.

M1-Nick
Level 3

๐Ÿ‘‹@ktgross15- sounds good! You know where to find me if you have questions!

๐Ÿ‘‹@ktgross15- sounds good! You know where to find me if you have questions!

AshleyW
Dataiker

Hi, 

Updating this thread to let you know that with the release of 12.5 we have a more fluid SQL recipe <--> Notebook experience: SQL recipes can now be edited in Notebooks, just like Python recipes can!

When you open a SQL recipe, you'll see a button, 'Edit in Notebook', that lets you do just that. 

Cheers, 

Ashley

Status changed to: Released

Hi, 

Updating this thread to let you know that with the release of 12.5 we have a more fluid SQL recipe <--> Notebook experience: SQL recipes can now be edited in Notebooks, just like Python recipes can!

When you open a SQL recipe, you'll see a button, 'Edit in Notebook', that lets you do just that. 

Cheers, 

Ashley