API SQL endpoint with Hive

[kuntaldeb]

Hi, 

As per latest DSS documentation https://doc.dataiku.com/dss/latest/apinode/endpoint-sql-query.html, SQL API endpoint on Hive is not supported. This also means Lookup API endpoint on a Hive Dataset under deployment policy "Referenced (SQL only)" is not supported as well.

Is there a plan to enable SQL endpoint for Hive tables and datasets?

- Kuntal.

[Clément_Stenac]

Hi,

"SQL query" endpoint with Hive is under consideration, but we have not made a determination on it nor have a date for it. Note that it may be achievable by creating a connection to Hive as a JDBC connection.

We will not implement Lookup endpoint with Hive and we would strongly advise you not to try to implement it as a JDBC connection. The reason is that Hive is an analytical database that is purely optimized for batch aggregations and scans. It is unsuitable for any kind of lookup-like activities that need a real database with indexes and fast lookup abilities

[kuntaldeb]

Hi,

Thanks for your response.

We have configured Hive connection through HiveServer2 configuration in DSS Settings and not via Other JDBC connections in Connections; hence, Hive DBs are not showing up under SQL connections in API SQL endpoints . We will explore on the JDBC approach and wait on news for Hive SQL endpoint.

Regarding lookup API, I agree with you strategy now; however, Hive is moving towards transnational approach and we might be in need of this functionality not very far in future.

- Kuntal.