API SQL endpoint with Hive

kuntaldeb

Hi,

As per latest DSS documentation https://doc.dataiku.com/dss/latest/apinode/endpoint-sql-query.html, SQL API endpoint on Hive is not supported. This also means Lookup API endpoint on a Hive Dataset under deployment policy "Referenced (SQL only)" is not supported as well.

Is there a plan to enable SQL endpoint for Hive tables and datasets?

- Kuntal.

Clément_Stenac

Hi,

"SQL query" endpoint with Hive is under consideration, but we have not made a determination on it nor have a date for it. Note that it may be achievable by creating a connection to Hive as a JDBC connection.

We will not implement Lookup endpoint with Hive and we would strongly advise you not to try to implement it as a JDBC connection. The reason is that Hive is an analytical database that is purely optimized for batch aggregations and scans. It is unsuitable for any kind of lookup-like activities that need a real database with indexes and fast lookup abilities

kuntaldeb

Hi,

Thanks for your response.

We have configured Hive connection through HiveServer2 configuration in DSS Settings and not via Other JDBC connections in Connections; hence, Hive DBs are not showing up under SQL connections in API SQL endpoints . We will explore on the JDBC approach and wait on news for Hive SQL endpoint.

Regarding lookup API, I agree with you strategy now; however, Hive is moving towards transnational approach and we might be in need of this functionality not very far in future.

- Kuntal.