Join: All tables are in SQL DB but Dataiku can't use SQL as execution engine

ifotopoulos

If a dataset is imported in Dataiku using "SQL query" as Mode it's not possible to use the SQL as execution engine for further operation even if all the tables involved are in the same SQL Database:

I think that since all the data is in the SQL database it should be able to use sql as the execution engine.

Clément_Stenac

Hi,

We confirm that this is the expected behavior of DSS. Only tables (or views, which can be read as tables) can be used as inputs of the "SQL query" recipe.

If needed, you can either create views or "cheat" by just having a single SQL table dataset as input but using other tables / queries / whatever in your query.

ifotopoulos

Thanks for the quick reply, that's good to know (I'm not sure it 's documented somewhere). I'm going to use views in the future for this kind of operations.

Just a side note, even though I aborted the job because it was too slow (since it was copying all the Data in Dataiku to do the join) , IT is complaining that there are cached tables/files in the Dataiku server that are consuming too much space.
It seems that even though I aborted the job, there was no proper clean-up of the data that it already brought on the server.