Improved SQL Experience

[M1-Nick]

It would be really nice if the SQL experience felt more like it was a natural part of the workflow.

Right now when you start a blank project, the flow experiences pushes you to first add a dataset. If that dataset is in a databases, you are asked to select your table to create your dataset. If you select writing with a custom SQL query, you are given a pretty distinct warning not to use this and a small box to enter some sql into. This most likely means you aren't starting here with SQL.

Another alternative for this is to just select a random table as the dataset so that you can then add a SQL recipe to your flow and work from there.

If you go to the notebook and start with a SQL notebook, you get a pretty nice SQL editor, with features like explain plans etc.

These experiences feel super disjointed and very confusing.

What would be nice is if these these experience were combined and we were able to start with the SQL notebook, write the SQL and then move from that view to back to the flow or into a Python/ R notebook.

It would also be great if we could take the SQL query, version control that into Github as a saved query for others to use.

[CoreyS]

Hi @M1-Nick thank you for your idea. I was wondering if you felt this was similar to @Marlan's idea of Transform the experience of writing SQL? If so I'd be happy to merge these two ideas together. 

[M1-Nick]

@CoreyS  I think they are similar yeah, but not the same.

@Marlantalks about doing jinja tempting and changing how the code is written.

I think my suggestion is more around how the various SQL options fit into the overall worksteam of Dataiku.  Basically shifting things around more and making the way the options are presented to the user more seamless.

I think combining them would be something that could be done if you want to completely redefine the SQL experience, otherwise they are could probably be tackled as separate requests.

[Marlan]

Hi @CoreyS, to weigh in here too... I agree with @M1-Nick that the ideas are different and should remain separate.

That said, I like these ideas. My typical flow in a new project is to start with a code recipe (SQL Script) and yes it would be great to be able to work between SQL recipes (probably most easily SQL Script recipes) and SQL Notebooks just like we can with Python recipes.

Marlan

[ktgross15]

Hey @M1-Nick ! Fancy meeting you here 🙂

Thanks for the feedback, we've logged it internally and will let you know if we have updates on any of these topics.

[M1-Nick]

👋@ktgross15- sounds good! You know where to find me if you have questions!